In this guide, we will show you how to calculate
F1 Score
for
YOLOv8
predictions in a few lines of code using the open source supervision Python package.
We will:
1. Install supervision
2. Load data
3. Calculate
F1 Score
for
YOLOv8
model detections.
Without further ado, let's get started!
First, install the supervision pip package:
pip install supervision
Once you have installed supervision, you are ready to load your data and start writing logic for your application.
In this guide, we are going to walk through an example of comparing the results from two models on a single image.
First, we need to load two models. We can then run the same image through both models and retrieve their detections. We will parse the detections into an sv.Detections object. sv.Detections allows us to represent all detections in a standard way that can be processed by the supervision metrics Python API.
To load models and run inference, use the following code:
data loader.
You can load data using the following code:
import supervision as sv
from supervision.metrics import MeanAveragePrecision
from inference import get_model
import matplotlib.pyplot as plt
# !wget https://media.roboflow.com/notebooks/examples/dog.jpeg
image = "dog.jpeg"
model_1 = get_model("yolov8n-640")
model_2 = get_model("yolov8s-640")
results_1 = model_1.infer(image)[0]
results_2 = model_2.infer(image)[0]
detections_1 = sv.Detections.from_inference(results_1)
detections_2 = sv.Detections.from_inference(results_2)
We now have predictions from two models from which we can calculate metrics.
We can calculate
F1 Score
using detections from two or more
YOLOv8
models (or, compare ground truth annotations to the results from a model) using the supervision metrics API. This API has support for calculating several metrics for populat computer vision models.
We can calculate
F1 Score
using the following code:
# MeanAveragePrecision().update([detections], [targets])
# in this example, we assume detections_2 contains the best detections (the largest model)
# if you are using the API with a ground truth dataset, detections_2 could be annotations from your dataset
# learn how to load annotations from a dataset with https://supervision.roboflow.com/latest/datasets/core/
map_n_metric = MeanAveragePrecision().update([detections_1], [detections_2]).compute()
print(map_n_metric.map50_95)
from supervision.metrics import F1Score
# .update() takes detections and targets
# in this example, we assume detections_2 contains the best detections (the largest model)
# if you are using the API with a ground truth dataset, detections_2 could be annotations from your dataset
# learn how to load annotations from a dataset with https://supervision.roboflow.com/latest/datasets/core/
f1_metric = F1Score()
f1_result = f1_metric.update(detections_1, detections_2).compute()
print(f1_result)
print(f1_result.f1_50)
print(f1_result.small_objects.f1_50)
supervision provides an extensive range of functionalities for working with computer vision models. With supervision, you can:
1. Process and filter detections and segmentation masks from a range of popular models (YOLOv5, Ultralytics YOLOv8, MMDetection, and more).
2. Process and filter classifications.
3. Compute confusion matrices.
And more! To learn about the full range of functionality in supervision, check out the supervision documentation.