How to Calculate Mean Average Precision from Detectron2 Detections

In this guide, we will show you how to calculate

Mean Average Precision (mAP)

for

predictions in a few lines of code using the open source supervision Python package.

We will:

1. Install supervision
2. Load data

3. Calculate

Mean Average Precision (mAP)

for

model detections.

Without further ado, let's get started!

Step #1: Install supervision

First, install the supervision pip package:

pip install supervision

Once you have installed supervision, you are ready to load your data and start writing logic for your application.

Step #2: Load Model(s)

In this guide, we are going to walk through an example of comparing the results from two models on a single image.

First, we need to load two models. We can then run the same image through both models and retrieve their detections. We will parse the detections into an sv.Detections object. sv.Detections allows us to represent all detections in a standard way that can be processed by the supervision metrics Python API.

To load models and run inference, use the following code:

data loader.

You can load data using the following code:
‍


import cv2
import supervision as sv
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg


image = cv2.imread("image.png")
cfg = get_cfg()
cfg.merge_from_file("path/to/config")
cfg.MODEL.WEIGHTS = "weights"
predictor_model_1 = DefaultPredictor(cfg)

cfg = get_cfg()
cfg.merge_from_file("path/to/config-2")
cfg.MODEL.WEIGHTS = "weights-2"
predictor_model_2 = DefaultPredictor(cfg)

result_1 = predictor_model_1(image)
detections_1 = sv.Detections.from_detectron2(result_1)

result_2 = predictor_model_2(image)
detections_2 = sv.Detections.from_detectron2(result_2)

We now have predictions from two models from which we can calculate metrics.

Step #3: Calculate Mean Average Precision (mAP)

We can calculate

Mean Average Precision (mAP)

using detections from two or more

models (or, compare ground truth annotations to the results from a model) using the supervision metrics API. This API has support for calculating several metrics for populat computer vision models.

We can calculate

Mean Average Precision (mAP)

using the following code:
‍


# MeanAveragePrecision().update([detections], [targets])
# in this example, we assume detections_2 contains the best detections (the largest model)
# if you are using the API with a ground truth dataset, detections_2 could be annotations from your dataset
# learn how to load annotations from a dataset with https://supervision.roboflow.com/latest/datasets/core/
map_n_metric = MeanAveragePrecision().update([detections_1], [detections_2]).compute()
print(map_n_metric.map50_95)


from supervision.metrics import F1Score

# .update() takes detections and targets
# in this example, we assume detections_2 contains the best detections (the largest model)
# if you are using the API with a ground truth dataset, detections_2 could be annotations from your dataset
# learn how to load annotations from a dataset with https://supervision.roboflow.com/latest/datasets/core/
f1_metric = F1Score()
f1_result = f1_metric.update(detections_1, detections_2).compute()

print(f1_result)
print(f1_result.f1_50)
print(f1_result.small_objects.f1_50)

Next steps

supervision provides an extensive range of functionalities for working with computer vision models. With supervision, you can:

1. Process and filter detections and segmentation masks from a range of popular models (YOLOv5, Ultralytics YOLOv8, MMDetection, and more).
2. Process and filter classifications.
3. Compute confusion matrices.

And more! To learn about the full range of functionality in supervision, check out the supervision documentation.

Write concise computer vision logic with supervision

How to Calculate Mean Average Precision from Detectron2 Detections

Step #1: Install supervision

Step #2: Load Model(s)

Step #3: Calculate Mean Average Precision (mAP)

Next steps

Join over 1 million developers building with Roboflow