Top Vision-Language Models

Segment Anything (SAM) is an image segmentation model developed by Meta Research, capable of doing zero-shot segmentation.

The YOLOv8 pose estimation model allows you to detect keypoints in an image.

YOLOv8 is a state-of-the-art object detection and image segmentation model created by Ultralytics, the developers of YOLOv5.

The state-of-the-art YOLOv8 model comes with support for instance segmentation tasks.

YOLOv9 is an object detection model architecture released on February 21st, 2024.

Grounding DINO is a zero-shot object detection model made by combining a Transformer-based DINO detector and grounded pre-training.

Segment Anything 2 (SAM 2) is a real-time image and video segmentation model.

YOLO-World is a zero-shot object detection model.

PaliGemma is a vision language model (VLM) by Google that has multimodal capabilities.

GPT-4o is OpenAI’s third major iteration of GPT-4 expanding on the capabilities of GPT-4 with Vision

Tesseract is a highly popular OCR engine and project, now primarily developed open-source.

YOLOv5 Instance Segmentation is a version of YOLOv5 that can be used for instance segmentation tasks.

YOLOv5 Classification is a version of the YOLOv5 model used in single-label and multi-label image classification.

A very fast and easy to use PyTorch model that achieves state of the art (or near state of the art) results.

YOLO11 is a computer vision model that you can use for object detection, segmentation, and classification.

Detectron2 is model zoo of it's own for computer vision models written in PyTorch.

You can retrieve bounding boxes whose edges match an angled object by training an oriented bounding boxes object detection model, such as YOLOv8's Oriented Bounding Boxes model.

Mask RCNN is a convolutional neural network for instance segmentation.

CLIP (Contrastive Language-Image Pre-Training) is an impressive multimodal zero-shot image classifier that achieves impressive results in a wide range of domains with no fine-tuning. It applies the recent advancements in large-scale transformers like GPT-3 to the vision arena.

An image classification model built using YOLOv8.

LLaVA is an open source multimodal language model that you can use for visual question answering and has limited support for object detection.

GroundedSAM combines Grounding DINO with the Segment Anything Model to identify and segment objects in an image given text captions.

Detection Transformer (DETR) is an end-to-end object detection model implemented using the Transformer architecture.

YOLOv7 is a state of the art object detection model.

YOLOv7 Instance Segmentation lets you perform segmentation tasks with the YOLOv7 model.

The Vision Transformer leverages powerful natural language processing embeddings (BERT) and applies them to images.

YOLOX is a high-performance object detection model.

YOLOv10 is a real-time object detection model introduced in the paper "YOLOv10: Real-Time End-to-End Object Detection".

One of the most accurate object detection algorithms but requires a lot of power at inference time. A good choice if you can do processing asynchronously on a server.

EfficientNet is from a family of image classification models from GoogleAI that train comparatively quickly on small amounts of data, making the most of limited datasets.

Though it is no longer the most accurate object detection algorithm, YOLO v3 is still a very good choice when you need real-time detection while maintaining excellent accuracy. PyTorch version.

Though it is no longer the most accurate object detection algorithm, YOLO v3 is still a very good choice when you need real-time detection while maintaining excellent accuracy. Keras implementation.

FastSAM is an image segmentation model trained using 2% of the data in the Segment Anything Model SA-1B dataset.

MT-YOLOv6 is a YOLO based model released in 2022.

Surya is a Python package designed for OCR on document layout analysis.

A simple, fully convolutional model for real-time instance segmentation

CogVLM shows strong performance in Visual Question Answering (VQA) and other vision tasks.

YOLOv4 has emerged as the best real time object detection model. YOLOv4 carries forward many of the research contributions of the YOLO family of models along with new modeling and data augmentation techniques. This implementation is in PyTorch.

YOLO-NAS is an object detection model developed by Deci that achieves SOTA performances compared to YOLOv5, v7, and v8.

YOLO-NAS Pose is a keypoint detection model developed by Deci AI.

ByteTrack is a multi-object tracking computer vision algorithm.

MMOCR is an Optical Character Recognition model zoo implemented with the MMDetection package.

Qwen-VL is an LMM developed by Alibaba Cloud. Qwen-VL accepts images, text, and bounding boxes as inputs. The model can output text and bounding boxes. Qwen-VL naturally supports English, Chinese, and multilingual conversation.

DocTR is an Optical Character Recognition tool powered by deep learning.

SegFormer is a computer vision framework used in semantic segmentation tasks, implemented with transformers.

Scaled YOLOv4 is an extension of the YOLOv4 research implemented in the YOLOv5 PyTorch framework.

YOLOR (You Only Learn One Representation) is an object detection model that uses both implicit and explicit knowledge to make predictions.

SigLIP is an image embedding model defined in the "Sigmoid Loss for Language Image Pre-Training" paper.

Detic is an open source segmentation model developed by Meta Research and released in 2022.

YOLOv5-OBB is a variant of YOLOv5 that supports oriented bounding boxes. This model is designed to yield predictions that better fit objects that are positioned at an angle.

OneFormer is a state-of-the-art multi-task image segmentation framework that is implemented using transformers.

MetaCLIP is a zero-shot classification and embedding model developed by Meta AI.

YOLOS looks at patches of an image to to form "patch tokens", which are used in place of the traditional wordpiece tokens in NLP.

The 4M model is a versatile multimodal Transformer model developed by EPFL and Apple, capable of handling a handful of vision and language tasks.

BakLLaVA is an LMM developed by LAION, Ontocord, and Skunkworks AI. BakLLaVA uses a Mistral 7B base augmented with the LLaVA 1.5 architecture.

L2CS-Net is a gaze estimation model that enables you to calculate where someone is looking and in what direction someone is looking.

This architecture provides good realtime results on limited compute. It's designed to run in realtime (30 frames per second) even on mobile devices.

CoDet is an open vocabulary zero-shot object detection model.

A fast, simple convolutional neural network that gets the job done for many tasks, including classification.

Grounded EdgeSAM is a combination of Grounding DINO, a zero-shot object detection model, and EdgeSAM, a fast zero-shot image segmentation model.

Use Grounding DINO, Segment Anything, and CLIP to label objects in images.

MobileNet is a GoogleAI model well-suited for on-device, real-time classification (distinct from MobileNetSSD, Single Shot Detector). This implementation leverages transfer learning from ImageNet to your dataset.

A fast, simple convolutional neural network that gets the job done for many tasks, including classification.

YOLOv4 has emerged as the best real time object detection model. YOLOv4 carries forward many of the research contributions of the YOLO family of models along with new modeling and data augmentation techniques. This implementation is in Darknet.

A scalable, state of the art object detection model, implemented here within the TensorFlow 2 Object Detection API.

EfficientDet achieves the best performance in the fewest training epochs among object detection model architectures, making it a highly scalable architecture especially when operating with limited compute.

The tiny and fast version of YOLOv4 - good for training and deployment on limited compute resources, and getting a feel for your dataset

RTMDet is an efficient real-time object detector, with self-reported metrics outperforming the YOLO series. It achieves 52.8% AP on COCO with 300+ FPS on an NVIDIA 3090 GPU, making it one of the fastest and most accurate object detectors available as of writing this post.

DINOv2 is a self-supervised method for training computer vision models developed by Meta Research and released in April 2023.

Kosmos-2 is a multimodal language model capable of object detection and grounding text in images.

OWLv2 is a transformer-based object detection model developed by Google Research. OWLv2 is the successor to OWL ViT.

FastViT is a fast image classification model developed by Apple.

OWL-ViT is a transformer-based object detection model developed by Google Research.

BLIPv2 is a multimodal model developed by Salesforce Research.

GPT-4 with Vision is a multimodal language model developed by OpenAI.

VLPart, developed by Meta Research, is an object detection and segmentation model that works with an open vocabulary

MobileCLIP is an image embedding model developed by Apple and introduced in the "MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training" paper

BioCLIP is a Vision Foundation Model for the Tree of Life

RemoteCLIP is a zero-shot classification model for remote sensing.

AltCLIP is a zero-shot image classification model.

Gemini is a family of Large Multimodal Models (LMMs) developed by Google Deepmind focused specifically on multimodality.

TrOCR is a Transformer-based OCR model developed by researchers from Microsoft Research.

You can use the set of PaliGemma weights trained on the COCO Captions dataset for zero-shot image captioning.

You can use the set of PaliGemma weights trained on the Screen2Words dataset for asking questions about website screenshots.

You can use the set of PaliGemma weights trained on the VQAv2 dataset for asking questions about the contents of images.

You can use the set of PaliGemma weights trained on the DocVQA dataset for asking questions about documents.

You can use the set of PaliGemma weights trained on the OCRVQA dataset for performing OCR on images.

Florence-2 is a lightweight vision-language model open-sourced by Microsoft under the MIT license.

Florence-2 OCR is a subset of Florence-2 that can read characters in images.

PaliGemma is a vision language model (VLM) by Google that has multimodal capabilities.

GPT-4o is OpenAI’s third major iteration of GPT-4 expanding on the capabilities of GPT-4 with Vision

CogVLM shows strong performance in Visual Question Answering (VQA) and other vision tasks.

Qwen-VL is an LMM developed by Alibaba Cloud. Qwen-VL accepts images, text, and bounding boxes as inputs. The model can output text and bounding boxes. Qwen-VL naturally supports English, Chinese, and multilingual conversation.

BakLLaVA is an LMM developed by LAION, Ontocord, and Skunkworks AI. BakLLaVA uses a Mistral 7B base augmented with the LLaVA 1.5 architecture.

Gemini is a family of Large Multimodal Models (LMMs) developed by Google Deepmind focused specifically on multimodality.