Deploy CLIP to a GPU

Set up your GPU machine and run CLIP with Roboflow Inference, on its own or inside a Workflow.

Overview

CLIP compares an image with text prompts and scores how well each prompt describes it, which makes it a zero-shot classifier: no training, just the labels you want to tell apart. To deploy CLIP to a GPU, you will:

  1. Set up your GPU machine and start the Inference Server
  2. Run CLIP on an image
  3. Run CLIP in a Workflow
  4. Deploy with a coding agent

Step 1: Set Up Your GPU Machine

You need a Linux machine with an NVIDIA GPU, the NVIDIA driver, Docker and the NVIDIA Container Toolkit (opens in a new tab), which lets containers use the GPU. Check that the driver sees the GPU:

Shell
nvidia-smi

Then install the Inference CLI and start the server. It finds the GPU and pulls the roboflow/roboflow-inference-server-gpu image:

Shell
pip install inference-cli
inference server start

To start the container yourself instead:

Shell
sudo docker run -d \
    --name inference-server \
    --gpus all \
    --read-only \
    -p 127.0.0.1:9001:9001 \
    --volume ~/.inference/cache:/tmp:rw \
    --security-opt="no-new-privileges" \
    --cap-drop="ALL" \
    --cap-add="NET_BIND_SERVICE" \
    roboflow/roboflow-inference-server-gpu:latest

The server now answers on port 9001 of this machine only. To call it from another machine, start it with inference server start --bind-address 0.0.0.0 and limit who can reach the port: the server has no authentication by default.

Step 2: Run CLIP on an Image

Shell
pip install inference-sdk

Set the ROBOFLOW_API_KEY environment variable to your Roboflow API key (opens in a new tab) first. Then run:

Python
import os

from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

result = client.clip_compare(
    subject="image.jpg",
    prompt=["a forklift", "a person", "an empty aisle"],
)
print(result["similarity"])

The result scores each prompt against the image; the highest score is the best match. The first request downloads the model weights, so it takes longer than the ones after. See the CLIP docs (opens in a new tab) for every option.

Step 3: Run a Workflow

A Workflow chains models with the logic around them, such as counting, tracking, filtering by zone or sending results to your own systems, and runs on your GPU machine like a single model does. Build one in the Roboflow app with the CLIP block (opens in a new tab), then run it by its workspace and Workflow ID:

Python
import os

from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)

result = client.run_workflow(
    workspace_name="YOUR_WORKSPACE",
    workflow_id="YOUR_WORKFLOW_ID",
    images={"image": "image.jpg"},
)
print(result[0])

The result has one entry per image, holding every output the Workflow defines. To run the Workflow on video, pass workflow and workspace to client.webrtc.stream() in place of model_id (video with Workflows (opens in a new tab)). See Deploy a Workflow (opens in a new tab) for the other ways to run one.

Step 4: Deploy with a Coding Agent

The Roboflow MCP server (opens in a new tab) gives a coding agent such as Claude Code or Cursor access to your Roboflow workspace. It can list your models, run one on a test image, build and run a Workflow, and write the deployment code for your GPU machine. Add it to Claude Code:

Shell
claude mcp add -s user roboflow --transport http https://mcp.roboflow.com/mcp

The first time the agent uses it, you sign in to Roboflow in your browser. In Cursor, add https://mcp.roboflow.com/mcp under mcpServers in ~/.cursor/mcp.json. Then ask the agent, for example: "Deploy my CLIP model to a GPU with Roboflow Inference and write a script that runs it on my camera stream."

Start where you are

Speak with an AI expert

Our team will help you start solving business problems on the first call.

  • Solution architecting
  • Live demonstration
  • Pricing and specifications
  • Feasibility assessment

Over 80,000 organizations build with Roboflow.

Rivian
Pella
Chobani
USG Corporation
BNSF Railway
American Woodmark
Outcomes
Patrick Industries
Peer Robotics