Convert annotation formats

How To Convert Unity Perception JSON to OpenAI CLIP Classification

In this guide, we show you how to convert data between the Unity Synthetic Data and CLIP formats for free. You can use your converted data to train OpenAI CLIP models and other models that support the CLIP format.

Unity Perception JSON

Unity Perception is a toolkit for generating datasets for computer vision by using 3D renders.

Below, learn the structure of Unity Perception JSON.

captures_000.jsonJSON
{
    "version": "0.0.1",
    "captures": [{
        "id": "b49f1317-f3ce-42e3-8201-b85b0bd1574b",
        "sequence_id": "96bf8ade-5d77-49a4-82e2-dc1ef567ab82",
        "step": 0,
        "timestamp": 10.0,
        "sensor": {
            "sensor_id": "edf45916-afec-9881-8d21-922241c54310",
            "ego_id": "1ceb788a-8155-4860-99da-2b406536f198",
            "modality": "camera",
            "translation": [
                0.0,
                0.0,
                0.0
            ],
            "rotation": [
                0.0,
                0.0,
                0.0,
                1.0
            ],
            "camera_intrinsic": [
                [
                    0.999999762,
                    0.0,
                    0.0
                ],
                [
                    0.0,
                    1.77777731,
                    0.0
                ],
                [
                    0.2,
                    0.266666651,
                    -1.0006001
                ]
            ]
        },
        "ego": {
            "ego_id": "1ceb788a-8155-4860-99da-2b406536f198",
            "translation": [
                -595.11,
                1175.75,
                487.64
            ],
            "rotation": [
                0.0778284,
                0.9212198,
                -0.246840149,
                0.290459484
            ],
            "velocity": null,
            "acceleration": null
        },
        "filename": "RGB2327d665-d71f-4c4f-8c77-66df8fa29202/rgb_2.png",
        "format": "PNG",
        "annotations": [{
            "id": "a6aef966-1fb3-4d59-91df-a44866e81038",
            "annotation_definition": "f9f22e05-443f-4602-a422-ebe4ea9b55cb",
            "values": [{
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 2,
                    "x": 875.0,
                    "y": 303.0,
                    "width": 151.0,
                    "height": 122.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 3,
                    "x": 1226.0,
                    "y": 382.0,
                    "width": 181.0,
                    "height": 130.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 1,
                    "x": 660.0,
                    "y": 402.0,
                    "width": 148.0,
                    "height": 112.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 5,
                    "x": 1003.0,
                    "y": 381.0,
                    "width": 148.0,
                    "height": 141.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 42,
                    "x": 831.0,
                    "y": 389.0,
                    "width": 230.0,
                    "height": 225.0
                }
            ]
        }]
    },
    {
        "id": "87182978-7b34-4157-85f2-9fa0b57af085",
        "sequence_id": "96bf8ade-5d77-49a4-82e2-dc1ef567ab82",
        "step": 1,
        "timestamp": 20.0,
        "sensor": {
            "sensor_id": "edf45916-afec-9881-8d21-922241c54310",
            "ego_id": "1ceb788a-8155-4860-99da-2b406536f198",
            "modality": "camera",
            "translation": [
                0.0,
                0.0,
                0.0
            ],
            "rotation": [
                0.0,
                0.0,
                0.0,
                1.0
            ],
            "camera_intrinsic": [
                [
                    0.999999762,
                    0.0,
                    0.0
                ],
                [
                    0.0,
                    1.77777731,
                    0.0
                ],
                [
                    0.2,
                    0.266666651,
                    -1.0006001
                ]
            ]
        },
        "ego": {
            "ego_id": "1ceb788a-8155-4860-99da-2b406536f198",
            "translation": [
                -595.11,
                1175.75,
                487.64
            ],
            "rotation": [
                0.0778284,
                0.9212198,
                -0.246840149,
                0.290459484
            ],
            "velocity": null,
            "acceleration": null
        },
        "filename": "RGB2327d665-d71f-4c4f-8c77-66df8fa29202/rgb_3.png",
        "format": "PNG",
        "annotations": [{
            "id": "94bffb82-7753-4d79-8294-66e2976d7362",
            "annotation_definition": "f9f22e05-443f-4602-a422-ebe4ea9b55cb",
            "values": [{
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 2,
                    "x": 875.0,
                    "y": 303.0,
                    "width": 151.0,
                    "height": 122.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 3,
                    "x": 1226.0,
                    "y": 382.0,
                    "width": 181.0,
                    "height": 130.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 1,
                    "x": 660.0,
                    "y": 402.0,
                    "width": 148.0,
                    "height": 112.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 5,
                    "x": 1003.0,
                    "y": 381.0,
                    "width": 149.0,
                    "height": 142.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 42,
                    "x": 831.0,
                    "y": 389.0,
                    "width": 230.0,
                    "height": 225.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 4,
                    "x": 1317.0,
                    "y": 455.0,
                    "width": 201.0,
                    "height": 166.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 33,
                    "x": 454.0,
                    "y": 462.0,
                    "width": 213.0,
                    "height": 216.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 13,
                    "x": 899.0,
                    "y": 504.0,
                    "width": 308.0,
                    "height": 286.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 16,
                    "x": 222.0,
                    "y": 501.0,
                    "width": 189.0,
                    "height": 290.0
                },
                {
                    "label_id": 4,
                    "label_name": "hard-hat",
                    "instance_id": 38,
                    "x": 295.0,
                    "y": 685.0,
                    "width": 113.0,
                    "height": 173.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 10,
                    "x": 40.0,
                    "y": 630.0,
                    "width": 202.0,
                    "height": 252.0
                },
                {
                    "label_id": 12,
                    "label_name": "car",
                    "instance_id": 46,
                    "x": 827.0,
                    "y": 712.0,
                    "width": 257.0,
                    "height": 229.0
                },
                {
                    "label_id": 1,
                    "label_name": "person",
                    "instance_id": 26,
                    "x": 1402.0,
                    "y": 810.0,
                    "width": 254.0,
                    "height": 187.0
                },
                {
                    "label_id": 13,
                    "label_name": "bicycle",
                    "instance_id": 7,
                    "x": 667.0,
                    "y": 641.0,
                    "width": 298.0,
                    "height": 394.0
                },
                {
                    "label_id": 5,
                    "label_name": "door",
                    "instance_id": 31,
                    "x": 1143.0,
                    "y": 839.0,
                    "width": 652.0,
                    "height": 241.0
                }
            ]
        }]
    }]
}

OpenAI CLIP Classification

CLIP is an impressive multimodal zero-shot image classifier that achieves impressive results in a wide range of domains with no fine-tuning.

Below, learn the structure of OpenAI CLIP Classification.

_tokenization.txt
An example picture from the Hard Hat dataset depicting a man wearing a hard-hat
An example picture from the Hard Hat dataset depicting several men wearing hard-hats
An example picture from the Hard Hat dataset depicting several people, some wearing hard hats
An example picture from the Hard Hat dataset depicting several people

Step 1: Create a free Roboflow public workspace

Roboflow is a universal conversion tool for computer vision annotation formats. The Public plan is the best way for those exploring personal projects, class assignments, and other experiments to try Roboflow. To convert your dataset, start by creating a free workspace on the Public plan.

Create Roboflow Workspace

Step 2: Upload your data into Roboflow

Once your account has been created, click Create New Project. (opens in a new tab)

Roboflow's create project form, with Object Detection selected as the project type

Upload your data to Roboflow by dragging and dropping your Unity Perception JSON images and annotations into the upload space.

Roboflow's upload page, with a drag-and-drop area for images, annotations, videos and PDFs

Step 3: Generate Dataset Version

Next, click "Generate New Version" to generate a new version of your dataset:

Roboflow's Generate a Dataset Version page, showing the source images and train/test split for a new version

You can then apply any preprocessing or augmentation steps to your dataset:

The augmentation step of a new dataset version, with 90° Rotate and Brightness added

Step 4: Export Dataset Version

After generating, you will be prompted to Export your dataset. You can choose to receive your dataset as a .zip file or a curl download link. Choose OpenAI CLIP Classification when asked in what format you want to export your data. You will see a dropdown with various options like this:

Roboflow's Download dialog, with the export format list open and COCO highlighted

Congratulations, you have successfully converted your dataset from Unity Perception JSON format to OpenAI CLIP Classification format!

Video Tutorial

Convert Computer Vision Annotation Formats Tutorial

Want to dive deeper into converting annotation formats with Roboflow? In the tutorial below, we explore how to convert your data in the Roboflow dashboard. We also discuss rejecting annotations and train-test-validation splits.

Frequently Asked Questions

Yes! It is free to convert Unity Perception JSON data into the OpenAI CLIP Classification format on the Roboflow platform.

If you have between a few and a few thousand images, converting data between these formats will be quick. But, the time it takes to convert between data formats increases with the more images you have.

Show guide

Convert Unity Perception JSON to other formats

Convert other formats to OpenAI CLIP Classification

Related

Start where you are

Speak with an AI expert

Our team will help you start solving business problems on the first call.

  • Solution architecting
  • Live demonstration
  • Pricing and specifications
  • Feasibility assessment

Over 16,000 organizations build with Roboflow.

Rivian
Pella
Chobani
USG Corporation
BNSF Railway
American Woodmark
Outcomes
Patrick Industries
Peer Robotics
Create Roboflow Workspace