Top Foundation Vision Models

Segment Anything (SAM) is an image segmentation model developed by Meta Research, capable of doing zero-shot segmentation.

Grounding DINO is a zero-shot object detection model made by combining a Transformer-based DINO detector and grounded pre-training.

YOLO-World is a zero-shot object detection model.

PaliGemma is a vision language model (VLM) by Google that has multimodal capabilities.

GPT-4o is OpenAI’s third major iteration of GPT-4 expanding on the capabilities of GPT-4 with Vision

CLIP (Contrastive Language-Image Pre-Training) is an impressive multimodal zero-shot image classifier that achieves impressive results in a wide range of domains with no fine-tuning. It applies the recent advancements in large-scale transformers like GPT-3 to the vision arena.

LLaVA is an open source multimodal language model that you can use for visual question answering and has limited support for object detection.

SigLIP is an image embedding model defined in the "Sigmoid Loss for Language Image Pre-Training" paper.

Detic is an open source segmentation model developed by Meta Research and released in 2022.

MetaCLIP is a zero-shot classification and embedding model developed by Meta AI.

The 4M model is a versatile multimodal Transformer model developed by EPFL and Apple, capable of handling a handful of vision and language tasks.

CoDet is an open vocabulary zero-shot object detection model.

Kosmos-2 is a multimodal language model capable of object detection and grounding text in images.

OWLv2 is a transformer-based object detection model developed by Google Research. OWLv2 is the successor to OWL ViT.

OWL-ViT is a transformer-based object detection model developed by Google Research.

GPT-4 with Vision is a multimodal language model developed by OpenAI.

MobileCLIP is an image embedding model developed by Apple and introduced in the "MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training" paper

Gemini is a family of Large Multimodal Models (LMMs) developed by Google Deepmind focused specifically on multimodality.