Overview

GroundedSAM combines Grounding DINO with the Segment Anything Model to identify and segment objects in an image given text captions.

Performance

Use This Model

First, install Autodistill and Autodistill Grounded SAM:


pip install autodistill-grounded-sam autodistill-yolov8

Then, run:


from autodistill_grounded_sam import GroundedSAM
from autodistill.detection import CaptionOntology
from autodistill.utils import plot
import cv2

# define an ontology to map class names to our GroundedSAM prompt
# the ontology dictionary has the format {caption: class}
# where caption is the prompt sent to the base model, and class is the label that will
# be saved for that caption in the generated annotations
# then, load the model
base_model = GroundedSAM(
    ontology=CaptionOntology(
        {
            "person": "person",
            "shipping container": "shipping container",
        }
    )
)

# run inference on a single image
results = base_model.predict("logistics.jpeg")

plot(
    image=cv2.imread("logistics.jpeg"),
    classes=base_model.ontology.classes(),
    detections=results
)
# label all images in a folder called `context_images`
base_model.label("./context_images", extension=".jpeg")

Label Data Automatically with Grounded SAM

You can automatically label a dataset using Grounded SAM with help from Autodistill, an open source package for training computer vision models. You can label a folder of images automatically with only a few lines of code. Below, see our tutorials that demonstrate how to use Grounded SAM to train a computer vision model.

Grounded SAM