Deploy SAM 3 to AWS EC2

Set up an EC2 instance and run SAM 3 with Roboflow Inference, on its own or inside a Workflow.

Overview

SAM 3 segments every instance of a concept you name in text, such as "hard hat" or "pallet", with no training. It needs a GPU. To deploy SAM 3 to AWS EC2, you will:

  1. Set up an EC2 instance and start the Inference Server
  2. Run SAM 3 on an image
  3. Run SAM 3 in a Workflow
  4. Deploy with a coding agent

Step 1: Set Up an EC2 Instance

The quickest route is the Inference CLI's cloud command. It uses SkyPilot (opens in a new tab) to launch a GPU instance in your AWS account with the Inference Server already running. Connect SkyPilot to your AWS account first (SkyPilot's cloud setup (opens in a new tab)), then run this on your own computer:

Shell
pip install inference "skypilot[aws]"
inference cloud deploy --provider aws --compute-type gpu

When it finishes, it prints the server's address, such as http://34.66.116.66:9001. Use that address as api_url in the code on this page. inference cloud status lists your deployments, and inference cloud undeploy shuts one down so it stops billing.

To use an instance you manage yourself instead, launch a GPU instance (a g4dn.xlarge with an NVIDIA T4 runs most models) from an AWS Deep Learning Base GPU AMI, which comes with the NVIDIA driver, Docker and the NVIDIA Container Toolkit, then SSH in and start the server on all of the instance's network interfaces:

Shell
pip install inference-cli
inference server start --bind-address 0.0.0.0

Then add an inbound rule to the instance's security group: Custom TCP, port 9001, with your own IP address as the source. The server has no authentication by default, so never open the port to the whole internet (securing a self-hosted server (opens in a new tab)).

Step 2: Run SAM 3 on an Image

Shell
pip install inference-sdk

Set the ROBOFLOW_API_KEY environment variable to your Roboflow API key (opens in a new tab) first. Run the code on your own computer with api_url set to your server's address, or on the server itself with http://localhost:9001. Then run:

Python
import os

from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(
    api_url="http://YOUR_SERVER_IP:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

result = client.sam3_concept_segment(
    "image.jpg",
    prompts=[{"type": "text", "text": "hard hat"}],
)
print(result)

The result holds a polygon and a confidence for every instance of the concept in the image. The first request downloads the model weights, so it takes longer than the ones after. See the SAM 3 docs (opens in a new tab) for every option.

Step 3: Run a Workflow

A Workflow chains models with the logic around them, such as counting, tracking, filtering by zone or sending results to your own systems, and runs on your EC2 instance like a single model does. Build one in the Roboflow app with the SAM 3 block (opens in a new tab), then run it by its workspace and Workflow ID:

Python
import os

from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(
    api_url="http://YOUR_SERVER_IP:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

result = client.run_workflow(
    workspace_name="YOUR_WORKSPACE",
    workflow_id="YOUR_WORKFLOW_ID",
    images={"image": "image.jpg"},
)
print(result[0])

The result has one entry per image, holding every output the Workflow defines. To run the Workflow on video, pass workflow and workspace to client.webrtc.stream() in place of model_id (video with Workflows (opens in a new tab)). See Deploy a Workflow (opens in a new tab) for the other ways to run one.

Step 4: Deploy with a Coding Agent

The Roboflow MCP server (opens in a new tab) gives a coding agent such as Claude Code or Cursor access to your Roboflow workspace. It can list your models, run one on a test image, build and run a Workflow, and write the deployment code for your EC2 instance. Add it to Claude Code:

Shell
claude mcp add -s user roboflow --transport http https://mcp.roboflow.com/mcp

The first time the agent uses it, you sign in to Roboflow in your browser. In Cursor, add https://mcp.roboflow.com/mcp under mcpServers in ~/.cursor/mcp.json. Then ask the agent, for example: "Deploy my SAM 3 model to AWS EC2 with Roboflow Inference and write a script that runs it on my camera stream."

Start where you are

Speak with an AI expert

Our team will help you start solving business problems on the first call.

  • Solution architecting
  • Live demonstration
  • Pricing and specifications
  • Feasibility assessment

Over 80,000 organizations build with Roboflow.

Rivian
Pella
Chobani
USG Corporation
BNSF Railway
American Woodmark
Outcomes
Patrick Industries
Peer Robotics