In this guide, we will show you how to calculate
Mean Average Precision (mAP)
for
predictions in a few lines of code using the open source supervision Python package.
We will:
1. Install supervision
2. Load data
3. Calculate
Mean Average Precision (mAP)
for
model detections.
Without further ado, let's get started!
First, install the supervision pip package:
pip install supervision
Once you have installed supervision, you are ready to load your data and start writing logic for your application.
In this guide, we are going to walk through an example of comparing the results from two models on a single image.
First, we need to load two models. We can then run the same image through both models and retrieve their detections. We will parse the detections into an sv.Detections object. sv.Detections allows us to represent all detections in a standard way that can be processed by the supervision metrics Python API.
To load models and run inference, use the following code:
data loader.
You can load data using the following code:
import cv2
import supervision as sv
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
image = cv2.imread("image.png")
cfg = get_cfg()
cfg.merge_from_file("path/to/config")
cfg.MODEL.WEIGHTS = "weights"
predictor_model_1 = DefaultPredictor(cfg)
cfg = get_cfg()
cfg.merge_from_file("path/to/config-2")
cfg.MODEL.WEIGHTS = "weights-2"
predictor_model_2 = DefaultPredictor(cfg)
result_1 = predictor_model_1(image)
detections_1 = sv.Detections.from_detectron2(result_1)
result_2 = predictor_model_2(image)
detections_2 = sv.Detections.from_detectron2(result_2)
We now have predictions from two models from which we can calculate metrics.
We can calculate
Mean Average Precision (mAP)
using detections from two or more
models (or, compare ground truth annotations to the results from a model) using the supervision metrics API. This API has support for calculating several metrics for populat computer vision models.
We can calculate
Mean Average Precision (mAP)
using the following code:
# MeanAveragePrecision().update([detections], [targets])
# in this example, we assume detections_2 contains the best detections (the largest model)
# if you are using the API with a ground truth dataset, detections_2 could be annotations from your dataset
# learn how to load annotations from a dataset with https://supervision.roboflow.com/latest/datasets/core/
map_n_metric = MeanAveragePrecision().update([detections_1], [detections_2]).compute()
print(map_n_metric.map50_95)
from supervision.metrics import F1Score
# .update() takes detections and targets
# in this example, we assume detections_2 contains the best detections (the largest model)
# if you are using the API with a ground truth dataset, detections_2 could be annotations from your dataset
# learn how to load annotations from a dataset with https://supervision.roboflow.com/latest/datasets/core/
f1_metric = F1Score()
f1_result = f1_metric.update(detections_1, detections_2).compute()
print(f1_result)
print(f1_result.f1_50)
print(f1_result.small_objects.f1_50)
supervision provides an extensive range of functionalities for working with computer vision models. With supervision, you can:
1. Process and filter detections and segmentation masks from a range of popular models (YOLOv5, Ultralytics YOLOv8, MMDetection, and more).
2. Process and filter classifications.
3. Compute confusion matrices.
And more! To learn about the full range of functionality in supervision, check out the supervision documentation.