How to Calculate f1 Score from PaddlePaddle Detections

In this guide, we will show you how to calculate

F1 Score

for

predictions in a few lines of code using the open source supervision Python package.

We will:

1. Install supervision
2. Load data

3. Calculate

F1 Score

for

model detections.

Without further ado, let's get started!

Step #1: Install supervision

First, install the supervision pip package:

pip install supervision

Once you have installed supervision, you are ready to load your data and start writing logic for your application.

Step #2: Load Model(s)

In this guide, we are going to walk through an example of comparing the results from two models on a single image.

First, we need to load two models. We can then run the same image through both models and retrieve their detections. We will parse the detections into an sv.Detections object. sv.Detections allows us to represent all detections in a standard way that can be processed by the supervision metrics Python API.

To load models and run inference, use the following code:

data loader.

You can load data using the following code:
‍


import supervision as sv
import paddle
from ppdet.engine import Trainer
from ppdet.core.workspace import load_config

weights = ()
config = ()

cfg = load_config(config)
trainer = Trainer(cfg, mode='test')
trainer.load_weights(weights)

weights = ()
config = ()

cfg_1 = load_config(config_1)
trainer_1 = Trainer(cfg_1, mode='test')
trainer_1.load_weights(weights_1)

paddledet_result_1 = trainer_1.predict([images])[0]
paddledet_result_2 = trainer_2.predict([images])[0]

detections_1 = sv.Detections.from_paddledet(paddledet_result_1)
detections_2 = sv.Detections.from_paddledet(paddledet_result_2)

We now have predictions from two models from which we can calculate metrics.

Step #3: Calculate F1 Score

We can calculate

F1 Score

using detections from two or more

models (or, compare ground truth annotations to the results from a model) using the supervision metrics API. This API has support for calculating several metrics for populat computer vision models.

We can calculate

F1 Score

using the following code:
‍


# MeanAveragePrecision().update([detections], [targets])
# in this example, we assume detections_2 contains the best detections (the largest model)
# if you are using the API with a ground truth dataset, detections_2 could be annotations from your dataset
# learn how to load annotations from a dataset with https://supervision.roboflow.com/latest/datasets/core/
map_n_metric = MeanAveragePrecision().update([detections_1], [detections_2]).compute()
print(map_n_metric.map50_95)


from supervision.metrics import F1Score

# .update() takes detections and targets
# in this example, we assume detections_2 contains the best detections (the largest model)
# if you are using the API with a ground truth dataset, detections_2 could be annotations from your dataset
# learn how to load annotations from a dataset with https://supervision.roboflow.com/latest/datasets/core/
f1_metric = F1Score()
f1_result = f1_metric.update(detections_1, detections_2).compute()

print(f1_result)
print(f1_result.f1_50)
print(f1_result.small_objects.f1_50)

Next steps

supervision provides an extensive range of functionalities for working with computer vision models. With supervision, you can:

1. Process and filter detections and segmentation masks from a range of popular models (YOLOv5, Ultralytics YOLOv8, MMDetection, and more).
2. Process and filter classifications.
3. Compute confusion matrices.

And more! To learn about the full range of functionality in supervision, check out the supervision documentation.

Write concise computer vision logic with supervision

How to Calculate f1 Score from PaddlePaddle Detections

Step #1: Install supervision

Step #2: Load Model(s)

Step #3: Calculate F1 Score

Next steps

Join over 1 million developers building with Roboflow