Deploy RF-DETR to a GPU
Set up your GPU machine, run RF-DETR on images and video with Roboflow Inference, and build it into a Workflow.
Overview
To deploy a RF-DETR model to a GPU, you will:
- Set up your GPU machine and start the Inference Server
- Run a pretrained RF-DETR model to check the setup
- Upload your RF-DETR weights to Roboflow
- Run your model on an image
- Run your model on video
- Run a Workflow that uses your model
- Deploy with a coding agent
Step 1: Set Up Your GPU Machine
You need a Linux machine with an NVIDIA GPU, the NVIDIA driver, Docker and the NVIDIA Container Toolkit (opens in a new tab), which lets containers use the GPU. Check that the driver sees the GPU:
nvidia-smiThen install the Inference CLI and start the server. It finds the GPU and pulls the roboflow/roboflow-inference-server-gpu image:
pip install inference-cli
inference server startTo start the container yourself instead:
sudo docker run -d \
--name inference-server \
--gpus all \
--read-only \
-p 127.0.0.1:9001:9001 \
--volume ~/.inference/cache:/tmp:rw \
--security-opt="no-new-privileges" \
--cap-drop="ALL" \
--cap-add="NET_BIND_SERVICE" \
roboflow/roboflow-inference-server-gpu:latestThe server now answers on port 9001 of this machine only. To call it from another machine, start it with inference server start --bind-address 0.0.0.0 and limit who can reach the port: the server has no authentication by default.
Step 2: Run RF-DETR on Your Device
Check the setup with RF-DETR trained on the Microsoft COCO dataset (opens in a new tab), which detects 80 everyday objects. Install the client libraries on the machine that calls the server:
pip install inference-sdk supervision pillowSet the ROBOFLOW_API_KEY environment variable to your Roboflow API key (opens in a new tab) first. Then run:
import os
from inference_sdk import InferenceHTTPClient
from PIL import Image
import supervision as sv
client = InferenceHTTPClient(
api_url="http://localhost:9001",
api_key=os.environ["ROBOFLOW_API_KEY"],
)
image = Image.open("image.jpg")
results = client.infer(image, model_id="rfdetr-small")
detections = sv.Detections.from_inference(results)
annotated = sv.BoxAnnotator().annotate(scene=image.copy(), detections=detections)
annotated = sv.LabelAnnotator().annotate(scene=annotated, detections=detections)
annotated.save("annotated.jpg")The first request downloads the model weights, so it takes longer than the ones after. annotated.jpg shows what RF-DETR found.
Step 3: Upload Your RF-DETR Weights
If you trained your RF-DETR model outside Roboflow, upload its weights to a Roboflow project to deploy it. (A model trained in Roboflow is ready already; skip to the next step.) You need a project with a dataset version: create a project, upload your images and generate a version. Then install the Roboflow Python package and run:
pip install roboflowimport os
from roboflow import Roboflow
rf = Roboflow(api_key=os.environ["ROBOFLOW_API_KEY"])
project = rf.workspace().project("YOUR_PROJECT_ID")
version = project.version(1)
version.deploy(
model_type="rfdetr-small",
model_path="output",
filename="checkpoint_best_total.pth",
)model_path is the folder your training run wrote, and filename the checkpoint inside it. Set model_type to the model you trained: rfdetr-nano to rfdetr-2xlarge. Replace YOUR_PROJECT_ID and the version number with your own (where to find them (opens in a new tab)). See Upload Custom Weights (opens in a new tab) for the training package versions each model needs.
Step 4: Run Your Model on an Image
Point the same code at your own model by its project ID and version:
import os
from inference_sdk import InferenceHTTPClient
from PIL import Image
import supervision as sv
client = InferenceHTTPClient(
api_url="http://localhost:9001",
api_key=os.environ["ROBOFLOW_API_KEY"],
)
image = Image.open("image.jpg")
results = client.infer(image, model_id="YOUR_PROJECT_ID/1")
detections = sv.Detections.from_inference(results)
annotated = sv.BoxAnnotator().annotate(scene=image.copy(), detections=detections)
annotated = sv.LabelAnnotator().annotate(scene=annotated, detections=detections)
annotated.save("annotated.jpg")sv.Detections.from_inference turns the response into detections you can count, filter, track or draw with supervision (opens in a new tab).
Step 5: Run Your Model on Video
Stream video to the server with the WebRTC client, which keeps one session open instead of sending a request per frame:
pip install "inference-sdk[webrtc]"import os
from inference_sdk import InferenceHTTPClient
from inference_sdk.webrtc import VideoFileSource
import supervision as sv
client = InferenceHTTPClient(
api_url="http://localhost:9001",
api_key=os.environ["ROBOFLOW_API_KEY"],
)
session = client.webrtc.stream(
source=VideoFileSource("video.mp4"),
model_id="YOUR_PROJECT_ID/1",
)
@session.on_frame
def on_frame(frame, data):
if data is None:
return
detections = sv.Detections.from_inference(data)
print(f"{len(detections)} objects in frame")
session.run()VideoFileSource reads a file. Use WebcamSource() for a camera on the machine running the script, or RTSPSource("rtsp://...") for a network camera the server can reach.
Step 6: Run a Workflow
A Workflow chains models with the logic around them, such as counting, tracking, filtering by zone or sending results to your own systems, and runs on your GPU machine like a single model does. Build one in the Roboflow app with your model in it, then run it by its workspace and Workflow ID:
import os
from inference_sdk import InferenceHTTPClient
client = InferenceHTTPClient(
api_url="http://localhost:9001",
api_key=os.environ["ROBOFLOW_API_KEY"],
)
result = client.run_workflow(
workspace_name="YOUR_WORKSPACE",
workflow_id="YOUR_WORKFLOW_ID",
images={"image": "image.jpg"},
)
print(result[0])The result has one entry per image, holding every output the Workflow defines. To run the Workflow on video, pass workflow and workspace to client.webrtc.stream() in place of model_id (video with Workflows (opens in a new tab)). See Deploy a Workflow (opens in a new tab) for the other ways to run one.
Step 7: Deploy with a Coding Agent
The Roboflow MCP server (opens in a new tab) gives a coding agent such as Claude Code or Cursor access to your Roboflow workspace. It can list your models, run one on a test image, build and run a Workflow, and write the deployment code for your GPU machine. Add it to Claude Code:
claude mcp add -s user roboflow --transport http https://mcp.roboflow.com/mcpThe first time the agent uses it, you sign in to Roboflow in your browser. In Cursor, add https://mcp.roboflow.com/mcp under mcpServers in ~/.cursor/mcp.json. Then ask the agent, for example: "Deploy my RF-DETR model to a GPU with Roboflow Inference and write a script that runs it on my camera stream."