Deploy YOLO26 to a GPU

Set up your GPU machine, run YOLO26 on images and video with Roboflow Inference, and build it into a Workflow.

Overview

To deploy a YOLO26 model to a GPU, you will:

  1. Set up your GPU machine and start the Inference Server
  2. Run a pretrained RF-DETR model to check the setup
  3. Upload your YOLO26 weights to Roboflow
  4. Run your model on an image
  5. Run your model on video
  6. Run a Workflow that uses your model
  7. Deploy with a coding agent

Step 1: Set Up Your GPU Machine

You need a Linux machine with an NVIDIA GPU, the NVIDIA driver, Docker and the NVIDIA Container Toolkit (opens in a new tab), which lets containers use the GPU. Check that the driver sees the GPU:

Shell
nvidia-smi

Then install the Inference CLI and start the server. It finds the GPU and pulls the roboflow/roboflow-inference-server-gpu image:

Shell
pip install inference-cli
inference server start

To start the container yourself instead:

Shell
sudo docker run -d \
    --name inference-server \
    --gpus all \
    --read-only \
    -p 127.0.0.1:9001:9001 \
    --volume ~/.inference/cache:/tmp:rw \
    --security-opt="no-new-privileges" \
    --cap-drop="ALL" \
    --cap-add="NET_BIND_SERVICE" \
    roboflow/roboflow-inference-server-gpu:latest

The server now answers on port 9001 of this machine only. To call it from another machine, start it with inference server start --bind-address 0.0.0.0 and limit who can reach the port: the server has no authentication by default.

Step 2: Run RF-DETR on Your Device

Check the setup with RF-DETR trained on the Microsoft COCO dataset (opens in a new tab), which detects 80 everyday objects. Install the client libraries on the machine that calls the server:

Shell
pip install inference-sdk supervision pillow

Set the ROBOFLOW_API_KEY environment variable to your Roboflow API key (opens in a new tab) first. Then run:

Python
import os

from inference_sdk import InferenceHTTPClient
from PIL import Image
import supervision as sv

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

image = Image.open("image.jpg")
results = client.infer(image, model_id="rfdetr-small")

detections = sv.Detections.from_inference(results)
annotated = sv.BoxAnnotator().annotate(scene=image.copy(), detections=detections)
annotated = sv.LabelAnnotator().annotate(scene=annotated, detections=detections)
annotated.save("annotated.jpg")

The first request downloads the model weights, so it takes longer than the ones after. annotated.jpg shows what RF-DETR found.

Step 3: Upload Your YOLO26 Weights

If you trained your YOLO26 model outside Roboflow, upload its weights to a Roboflow project to deploy it. (A model trained in Roboflow is ready already; skip to the next step.) You need a project with a dataset version: create a project, upload your images and generate a version. Then install the Roboflow Python package and run:

Shell
pip install roboflow
Python
import os

from roboflow import Roboflow

rf = Roboflow(api_key=os.environ["ROBOFLOW_API_KEY"])
project = rf.workspace().project("YOUR_PROJECT_ID")
version = project.version(1)

version.deploy(
    model_type="yolo26n",
    model_path="runs/detect/train",
)

model_path is the folder your training run wrote. Set model_type to the size you trained (n, s, m, l or x). Replace YOUR_PROJECT_ID and the version number with your own (where to find them (opens in a new tab)). See Upload Custom Weights (opens in a new tab) for the training package versions each model needs.

Step 4: Run Your Model on an Image

Point the same code at your own model by its project ID and version:

Python
import os

from inference_sdk import InferenceHTTPClient
from PIL import Image
import supervision as sv

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

image = Image.open("image.jpg")
results = client.infer(image, model_id="YOUR_PROJECT_ID/1")

detections = sv.Detections.from_inference(results)
annotated = sv.BoxAnnotator().annotate(scene=image.copy(), detections=detections)
annotated = sv.LabelAnnotator().annotate(scene=annotated, detections=detections)
annotated.save("annotated.jpg")

sv.Detections.from_inference turns the response into detections you can count, filter, track or draw with supervision (opens in a new tab).

Step 5: Run Your Model on Video

Stream video to the server with the WebRTC client, which keeps one session open instead of sending a request per frame:

Shell
pip install "inference-sdk[webrtc]"
Python
import os

from inference_sdk import InferenceHTTPClient
from inference_sdk.webrtc import VideoFileSource
import supervision as sv

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

session = client.webrtc.stream(
    source=VideoFileSource("video.mp4"),
    model_id="YOUR_PROJECT_ID/1",
)

@session.on_frame
def on_frame(frame, data):
    if data is None:
        return
    detections = sv.Detections.from_inference(data)
    print(f"{len(detections)} objects in frame")

session.run()

VideoFileSource reads a file. Use WebcamSource() for a camera on the machine running the script, or RTSPSource("rtsp://...") for a network camera the server can reach.

Step 6: Run a Workflow

A Workflow chains models with the logic around them, such as counting, tracking, filtering by zone or sending results to your own systems, and runs on your GPU machine like a single model does. Build one in the Roboflow app with your model in it, then run it by its workspace and Workflow ID:

Python
import os

from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

result = client.run_workflow(
    workspace_name="YOUR_WORKSPACE",
    workflow_id="YOUR_WORKFLOW_ID",
    images={"image": "image.jpg"},
)
print(result[0])

The result has one entry per image, holding every output the Workflow defines. To run the Workflow on video, pass workflow and workspace to client.webrtc.stream() in place of model_id (video with Workflows (opens in a new tab)). See Deploy a Workflow (opens in a new tab) for the other ways to run one.

Step 7: Deploy with a Coding Agent

The Roboflow MCP server (opens in a new tab) gives a coding agent such as Claude Code or Cursor access to your Roboflow workspace. It can list your models, run one on a test image, build and run a Workflow, and write the deployment code for your GPU machine. Add it to Claude Code:

Shell
claude mcp add -s user roboflow --transport http https://mcp.roboflow.com/mcp

The first time the agent uses it, you sign in to Roboflow in your browser. In Cursor, add https://mcp.roboflow.com/mcp under mcpServers in ~/.cursor/mcp.json. Then ask the agent, for example: "Deploy my YOLO26 model to a GPU with Roboflow Inference and write a script that runs it on my camera stream."

Start where you are

Speak with an AI expert

Our team will help you start solving business problems on the first call.

  • Solution architecting
  • Live demonstration
  • Pricing and specifications
  • Feasibility assessment

Over 80,000 organizations build with Roboflow.

Rivian
Pella
Chobani
USG Corporation
BNSF Railway
American Woodmark
Outcomes
Patrick Industries
Peer Robotics