Deploy SAM 3 to a GPU

Set up your GPU machine and run SAM 3 with Roboflow Inference, on its own or inside a Workflow.

Overview

SAM 3 segments every instance of a concept you name in text, such as "hard hat" or "pallet", with no training. It needs a GPU. To deploy SAM 3 to a GPU, you will:

  1. Set up your GPU machine and start the Inference Server
  2. Run SAM 3 on an image
  3. Run SAM 3 in a Workflow
  4. Deploy with a coding agent

Step 1: Set Up Your GPU Machine

You need a Linux machine with an NVIDIA GPU, the NVIDIA driver, Docker and the NVIDIA Container Toolkit (opens in a new tab), which lets containers use the GPU. Check that the driver sees the GPU:

Shell
nvidia-smi

Then install the Inference CLI and start the server. It finds the GPU and pulls the roboflow/roboflow-inference-server-gpu image:

Shell
pip install inference-cli
inference server start

To start the container yourself instead:

Shell
sudo docker run -d \
    --name inference-server \
    --gpus all \
    --read-only \
    -p 127.0.0.1:9001:9001 \
    --volume ~/.inference/cache:/tmp:rw \
    --security-opt="no-new-privileges" \
    --cap-drop="ALL" \
    --cap-add="NET_BIND_SERVICE" \
    roboflow/roboflow-inference-server-gpu:latest

The server now answers on port 9001 of this machine only. To call it from another machine, start it with inference server start --bind-address 0.0.0.0 and limit who can reach the port: the server has no authentication by default.

Step 2: Run SAM 3 on an Image

Shell
pip install inference-sdk

Set the ROBOFLOW_API_KEY environment variable to your Roboflow API key (opens in a new tab) first. Then run:

Python
import os

from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

result = client.sam3_concept_segment(
    "image.jpg",
    prompts=[{"type": "text", "text": "hard hat"}],
)
print(result)

The result holds a polygon and a confidence for every instance of the concept in the image. The first request downloads the model weights, so it takes longer than the ones after. See the SAM 3 docs (opens in a new tab) for every option.

Step 3: Run a Workflow

A Workflow chains models with the logic around them, such as counting, tracking, filtering by zone or sending results to your own systems, and runs on your GPU machine like a single model does. Build one in the Roboflow app with the SAM 3 block (opens in a new tab), then run it by its workspace and Workflow ID:

Python
import os

from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

result = client.run_workflow(
    workspace_name="YOUR_WORKSPACE",
    workflow_id="YOUR_WORKFLOW_ID",
    images={"image": "image.jpg"},
)
print(result[0])

The result has one entry per image, holding every output the Workflow defines. To run the Workflow on video, pass workflow and workspace to client.webrtc.stream() in place of model_id (video with Workflows (opens in a new tab)). See Deploy a Workflow (opens in a new tab) for the other ways to run one.

Step 4: Deploy with a Coding Agent

The Roboflow MCP server (opens in a new tab) gives a coding agent such as Claude Code or Cursor access to your Roboflow workspace. It can list your models, run one on a test image, build and run a Workflow, and write the deployment code for your GPU machine. Add it to Claude Code:

Shell
claude mcp add -s user roboflow --transport http https://mcp.roboflow.com/mcp

The first time the agent uses it, you sign in to Roboflow in your browser. In Cursor, add https://mcp.roboflow.com/mcp under mcpServers in ~/.cursor/mcp.json. Then ask the agent, for example: "Deploy my SAM 3 model to a GPU with Roboflow Inference and write a script that runs it on my camera stream."

Start where you are

Speak with an AI expert

Our team will help you start solving business problems on the first call.

  • Solution architecting
  • Live demonstration
  • Pricing and specifications
  • Feasibility assessment

Over 80,000 organizations build with Roboflow.

Rivian
Pella
Chobani
USG Corporation
BNSF Railway
American Woodmark
Outcomes
Patrick Industries
Peer Robotics