Overview

SigLIP is an image embedding model defined in the "Sigmoid Loss for Language Image Pre-Training" paper. Released in March, 2023, SigLIP uses CLIP’s framework with one twist: its loss function. Through this change, SigLIP achieves significant improvements in zero shot detections.

You can use SigLIP to calculate image embeddings. These embeddings can be used for:

Zero-shot image classification;
Calculating image similarity;
Deduplicating images in a dataset, and more.

Performance

‍

SigLIP achieves superior zero-shot performance on benchmarks like ImageNet, outperforming models like CLIP. The model’s design and training methodology make it particularly effective in generalizing to unseen data, a key advantage in many real-world applications

‍

Use This Model

First, install Autodistill and Autodistill SigLIP:


pip install autodistill autodistill-siglip

Then, run:


from autodistill_siglip import SigLIP
from autodistill.detection import CaptionOntology

# define an ontology to map class names to our SigLIP prompt
# the ontology dictionary has the format {caption: class}
# where caption is the prompt sent to the base model, and class is the label that will
# be saved for that caption in the generated annotations
# then, load the model
labels = ["person", "a forklift"]
base_model = SigLIP(
    ontology=CaptionOntology({item: item for item in labels})
)

results = base_model.predict("image.jpeg", confidence=0.1)

top_1 = results.get_top_k(1)

# show top label
print(labels[top_1[0][0]])

# label folder of images
base_model.label("./context_images", extension=".jpeg")

‍

Label Data Automatically with SigLIP

You can automatically label a dataset using SigLIP with help from Autodistill, an open source package for training computer vision models. You can label a folder of images automatically with only a few lines of code. Below, see our tutorials that demonstrate how to use SigLIP to train a computer vision model.

No items found.