HOW TO GUIDE

How to create a PaliGemma confusion matrix

Learn how to calculate confusion matrices for model results using the open source supervision Python package.

Overview

Evaluation is an essential part of the computer vision model development process. While you are striving to build the first version of your model, model evaluation will help you understand baseline performance and judge how close your model is to being ready for production. When working on future versions of a model, evaluation helps you understand the impact of each change you make.

One key part of evaluating models is computing confusion matrices. A confusion matrix is a visualization that shows how your model is performing on the classes on which it was trained.

In this guide, we are going to show you how to use the open source supervision Python package to create a

confusion matrix.

We will:
‍
1. Install supervision
2. Run inference on a dataset using a

PaliGemma

model
3. Create and plot a confusion matrix for the model

Without further ado, let's get started!

PaliGemma and Image Annotation Resources

Explore these resources to enhance your understanding of PaliGemma and image annotation techniques.

Step 1

Install supervision

First, install the supervision pip package:

pip install supervision

Once you have installed supervision, you are ready to load your data and start writing logic to filter detections.

Step 2

Load Data and Compute Matrix

First, we are going to load our dataset into a supervision.DetectionDataset() object. This object will contain information about all the images in a dataset. You can load datasets from many different model types, from YOLO to MMDetection. For this guide, we will use the

PaliGemma

data loader.

model.

We will use that callback to run inference on every image in our dataset, and compute a confusion matrix that shows how the model performs on the dataset.

Create a new Python file and add the following code:
‍


import supervision as sv

dataset = sv.DetectionDataset.from_yolo(...)

def callback(image: np.ndarray) -> sv.Detections:
    paligemma_result = " object"
    return sv.Detections.from_lmm(
        sv.LMM.PALIGEMMA,
        paligemma_result,
        resolution_wh=(1000, 1000),
        classes=[...]
    )

confusion_matrix = sv.ConfusionMatrix.benchmark(
   dataset = dataset,
   callback = callback
)

confusion_matrix.plot()

Set the DATASET value as the path to the folder where your dataset is stored.

Then, run the code to create the confusion matrix.

Step 3

Plot Confusion Matrix

We can plot the confusion matrix showing the results of the

PaliGemma

model evaluation using the following line of code


confusion_matrix.plot()

Step 4

Next Steps

supervision provides an extensive range of functionalities for working with computer vision models. With supervision, you can:

1. Process and filter detections and segmentation masks from a range of popular models (YOLOv5, Ultralytics YOLOv8, MMDetection, and more).
2. Process and filter classifications.
3. Plot bounding boxes and segmentation masks.

And more! To learn about the full range of functionality in supervision, check out the supervision documentation.