Deploy RF-DETR to NVIDIA Jetson

Set up your Jetson, run RF-DETR on images and video with Roboflow Inference, and build it into a Workflow.

Overview

To deploy a RF-DETR model to NVIDIA Jetson, you will:

  1. Set up your Jetson and start the Inference Server
  2. Run a pretrained RF-DETR model to check the setup
  3. Upload your RF-DETR weights to Roboflow
  4. Run your model on an image
  5. Run your model on video
  6. Run a Workflow that uses your model
  7. Deploy with a coding agent

Step 1: Set Up Your Jetson

Flash the Jetson with JetPack 5, 6 or 7 and keep at least 10 GB of disk free for the Inference image. An Orin NX 16 GB or larger runs bigger models best. JetPack includes NVIDIA's container runtime; install Docker (opens in a new tab) if the device does not have it yet.

Install the Inference CLI and start the server. It reads your JetPack version and pulls the image built for it, such as roboflow/roboflow-inference-server-jetson-6.2.0 on JetPack 6.2:

Shell
pip install inference-cli
inference server start

To start the container yourself on JetPack 6.2 instead:

Shell
sudo docker run -d \
    --name inference-server \
    --runtime nvidia \
    --read-only \
    -p 9001:9001 \
    --volume ~/.inference/cache:/tmp:rw \
    --security-opt="no-new-privileges" \
    --cap-drop="ALL" \
    --cap-add="NET_BIND_SERVICE" \
    roboflow/roboflow-inference-server-jetson-6.2.0:latest

On a Jetson the server answers on every network interface (port 9001), so other machines on your network can call it. Start it with inference server start --bind-address 127.0.0.1 to keep it on the device. The Jetson install guide (opens in a new tab) lists the image for every JetPack version.

Step 2: Run RF-DETR on Your Device

Check the setup with RF-DETR trained on the Microsoft COCO dataset (opens in a new tab), which detects 80 everyday objects. Install the client libraries on the machine that calls the server:

Shell
pip install inference-sdk supervision pillow

Set the ROBOFLOW_API_KEY environment variable to your Roboflow API key (opens in a new tab) first. Then run:

Python
import os

from inference_sdk import InferenceHTTPClient
from PIL import Image
import supervision as sv

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

image = Image.open("image.jpg")
results = client.infer(image, model_id="rfdetr-small")

detections = sv.Detections.from_inference(results)
annotated = sv.BoxAnnotator().annotate(scene=image.copy(), detections=detections)
annotated = sv.LabelAnnotator().annotate(scene=annotated, detections=detections)
annotated.save("annotated.jpg")

The first request downloads the model weights, so it takes longer than the ones after. annotated.jpg shows what RF-DETR found.

Step 3: Upload Your RF-DETR Weights

If you trained your RF-DETR model outside Roboflow, upload its weights to a Roboflow project to deploy it. (A model trained in Roboflow is ready already; skip to the next step.) You need a project with a dataset version: create a project, upload your images and generate a version. Then install the Roboflow Python package and run:

Shell
pip install roboflow
Python
import os

from roboflow import Roboflow

rf = Roboflow(api_key=os.environ["ROBOFLOW_API_KEY"])
project = rf.workspace().project("YOUR_PROJECT_ID")
version = project.version(1)

version.deploy(
    model_type="rfdetr-small",
    model_path="output",
    filename="checkpoint_best_total.pth",
)

model_path is the folder your training run wrote, and filename the checkpoint inside it. Set model_type to the model you trained: rfdetr-nano to rfdetr-2xlarge. Replace YOUR_PROJECT_ID and the version number with your own (where to find them (opens in a new tab)). See Upload Custom Weights (opens in a new tab) for the training package versions each model needs.

Step 4: Run Your Model on an Image

Point the same code at your own model by its project ID and version:

Python
import os

from inference_sdk import InferenceHTTPClient
from PIL import Image
import supervision as sv

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

image = Image.open("image.jpg")
results = client.infer(image, model_id="YOUR_PROJECT_ID/1")

detections = sv.Detections.from_inference(results)
annotated = sv.BoxAnnotator().annotate(scene=image.copy(), detections=detections)
annotated = sv.LabelAnnotator().annotate(scene=annotated, detections=detections)
annotated.save("annotated.jpg")

sv.Detections.from_inference turns the response into detections you can count, filter, track or draw with supervision (opens in a new tab).

Step 5: Run Your Model on Video

Stream video to the server with the WebRTC client, which keeps one session open instead of sending a request per frame:

Shell
pip install "inference-sdk[webrtc]"
Python
import os

from inference_sdk import InferenceHTTPClient
from inference_sdk.webrtc import VideoFileSource
import supervision as sv

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

session = client.webrtc.stream(
    source=VideoFileSource("video.mp4"),
    model_id="YOUR_PROJECT_ID/1",
)

@session.on_frame
def on_frame(frame, data):
    if data is None:
        return
    detections = sv.Detections.from_inference(data)
    print(f"{len(detections)} objects in frame")

session.run()

VideoFileSource reads a file. Use WebcamSource() for a camera on the machine running the script, or RTSPSource("rtsp://...") for a network camera the server can reach.

Step 6: Run a Workflow

A Workflow chains models with the logic around them, such as counting, tracking, filtering by zone or sending results to your own systems, and runs on your Jetson like a single model does. Build one in the Roboflow app with your model in it, then run it by its workspace and Workflow ID:

Python
import os

from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

result = client.run_workflow(
    workspace_name="YOUR_WORKSPACE",
    workflow_id="YOUR_WORKFLOW_ID",
    images={"image": "image.jpg"},
)
print(result[0])

The result has one entry per image, holding every output the Workflow defines. To run the Workflow on video, pass workflow and workspace to client.webrtc.stream() in place of model_id (video with Workflows (opens in a new tab)). See Deploy a Workflow (opens in a new tab) for the other ways to run one.

Step 7: Deploy with a Coding Agent

The Roboflow MCP server (opens in a new tab) gives a coding agent such as Claude Code or Cursor access to your Roboflow workspace. It can list your models, run one on a test image, build and run a Workflow, and write the deployment code for your Jetson. Add it to Claude Code:

Shell
claude mcp add -s user roboflow --transport http https://mcp.roboflow.com/mcp

The first time the agent uses it, you sign in to Roboflow in your browser. In Cursor, add https://mcp.roboflow.com/mcp under mcpServers in ~/.cursor/mcp.json. Then ask the agent, for example: "Deploy my RF-DETR model to NVIDIA Jetson with Roboflow Inference and write a script that runs it on my camera stream."

Start where you are

Speak with an AI expert

Our team will help you start solving business problems on the first call.

  • Solution architecting
  • Live demonstration
  • Pricing and specifications
  • Feasibility assessment

Over 80,000 organizations build with Roboflow.

Rivian
Pella
Chobani
USG Corporation
BNSF Railway
American Woodmark
Outcomes
Patrick Industries
Peer Robotics