Train a Computer Vision Model in ChatGPT with Roboflow

A defect detection model can find obvious paint damage and still miss the faint scratches an inspector needs to catch. Building a useful inspection system means understanding where the model succeeds, where it fails, and what to change next.

With the Roboflow plugin in ChatGPT (opens in a new tab), you can explore datasets, train computer vision models, and evaluate their predictions through a conversation. Ask questions about your labels, inspect missed detections, or compare training runs. Roboflow’s training and evaluation skill helps guide those decisions, while the connected tools perform the work in your workspace.

In this guide, we’ll train an RF-DETR model (opens in a new tab) to detect scratches and paint damage on alloy wheels. We’ll start with a labeled dataset from Roboflow Universe (opens in a new tab), prepare a dataset version, and train a baseline model. Then we’ll examine its errors, explore confidence thresholds, and use the results to choose and evaluate our next improvement.

Connect the Roboflow ChatGPT Plugin and Select a Project

Connect the Roboflow plugin in ChatGPT and sign in to your Roboflow account. Confirm the workspace you intend to use, especially if your account belongs to more than one. Training consumes Roboflow credits. Check your workspace’s training access and review the credits documentation (opens in a new tab) before launching a run. If you already have a relevant project, identify it now. For the public dataset used below, review its source images and annotations before deciding to fork it into the selected workspace. If you are starting with your own data, ChatGPT also supports image uploads and annotations. This tutorial uses Universe to provide a shared, labeled starting point.

Use the roboflow-training-and-evaluation Skill

The plugin combines live Roboflow operations with specialized guidance called skills. The roboflow-training-and-evaluation skill helps structure tasks such as choosing a model, interpreting evaluation results, diagnosing weak performance, and planning improvements.

A skill provides guidance. The connected tools perform supported actions in your Roboflow workspace. Naming a skill does not automatically start a training job or unlock features that your workspace cannot access. For this tutorial, the training skill works alongside:

• roboflow-universe for finding and examining public datasets.
• roboflow-data-management for understanding projects, labels, splits, and dataset versions.
• draw-vision for visualizing annotations or predictions when the required images and coordinates are available.

You can read more in Roboflow’s Agent Skills repository (opens in a new tab) and MCP documentation (opens in a new tab).

Explore Your Dataset Before Training

If you do not yet have a dataset URL, start by describing your inspection task. Ask the plugin to find relevant public datasets and return the information you need to compare them.

@Roboflow, search Roboflow Universe for labeled datasets suitable for training an object detection model to detect scratches and paint damage on alloy wheels.

Show up to five relevant datasets in a table with:
- Dataset name and owner
- Universe dataset URL
- Full source project ID (workspace/project)
- Task type
- Image count
- Class names
- Available dataset versions
- License

Recommend one for this training tutorial and briefly explain why. Mark information you cannot verify as “Not available.”

You'll see this output:

Compare the returned datasets and choose one for closer inspection. Use its Universe dataset URL in the next prompt to review sample images and annotations before forking it.

For this tutorial, the following steps use the public alloy-wheel-defect dataset on Roboflow Universe (opens in a new tab). If you select a different dataset, replace the URL and class names in the example prompts with those returned for your selection.

Begin by examining the dataset:

@Roboflow, inspect this Universe dataset: [DATASET_URL]. Summarize its task, classes, image count, and license. Retrieve representative images and annotations for scratch and paint-damage. Identify questions we should resolve before using it for a manufacturing inspection example.

This will give you a detailed information as shown in image below:

The ground truth image looks like this:

Look for whether the labels match your intended task. For instance, does a box surround the damaged region, or the entire wheel? Are similar defects labeled consistently? Do the examples include the lighting and surface finishes you expect to encounter?

The Universe image-exploration guide (opens in a new tab) explains how images and annotations are organized.

Bring the Selected Dataset into your Workspace

After reviewing and selecting a dataset, fork its images and annotations into your connected Roboflow workspace (opens in a new tab). This gives you a workspace project in which to prepare a dataset version and train your model. If your workspace already owns the selected project, reuse that project.

Replace [DATASET_URL] below with the Universe URL returned by the search:

@Roboflow, bring the selected Universe dataset at [DATASET_URL] into my connected Roboflow workspace for training.

If my workspace already owns this project, reuse it. Otherwise, fork the dataset and check the import task until it finishes.

Return:
- Project name
- Destination workspace
- Full project ID (workspace/project)
- Project link
- Image count and class names
- A summary of the images and annotations copied, or the existing project reused

Do not start training yet.

Wait for the import to finish before running the workspace health check. A new fork copies the images and annotations; source dataset versions and export settings are not carried over. You will generate your own dataset version in the next section.

Use the full project ID returned for your workspace as [PROJECT_ID] in the health-check, version-generation, and training prompts. The source project ID from the Universe search identifies the public dataset; the returned workspace project ID identifies the project you will work on.

Check the Dataset in your Workspace

With the project available in your workspace, examine its dataset health before preparing a version. Ask for a Dataset Health Check (opens in a new tab):

@Roboflow, inspect the health of [PROJECT_ID]. Report image and annotation counts, class distribution, train/validation/test counts, image dimensions, and missing or null annotations where available. Show which findings come from the health report and which require image review. Do not change the dataset yet.

Here's the response you'll get for the selected alloy wheel dataset.

Keep image counts separate from annotation counts. One wheel image may contain several labeled defects. Also distinguish an intentionally labeled defect-free image from an image that has simply not been annotated. Both may have zero boxes, but they have different meanings for training.

For the wheel example, investigate four questions:

For manufacturing data, keep images of the same physical wheel or closely related capture sequence together where those relationships are known. Use part identifiers, timestamps, or batch metadata to investigate leakage. A health report alone cannot prove that the splits are independent. Use training images to learn model weights, validation images to guide experiments, and the test set for a final assessment after your choices are fixed.

Prepare a Dataset Version

A dataset version (opens in a new tab) records a particular prepared dataset. It gives you a stable reference when comparing training runs. For this walkthrough, start with a deliberately simple preparation plan:

The proposed resize corresponds to Roboflow’s Fit (black edges) in option. Its preprocessing documentation (opens in a new tab) explains the available resize methods. Here, 512 × 512 is a starting point for RF-DETR Small, whose documented default resolution is 512. It is not a guarantee that tiny scratches will retain enough detail. Review representative resized examples before accepting the choice. See training resolutions by model type (opens in a new tab).

@Roboflow, prepare a baseline version plan for [PROJECT_ID]: Auto-Orient, preserve color, resize to 512 × 512 using Fit (black edges) in, and add no dataset augmentations. Preserve the reviewed split assignments. Check compatibility with RF-DETR Small and help me assess whether small defects remain visible. Present the plan before generating the version.

This will give you nice detailed information.

The baseline preview image looks like this:

Once satisfied, generate the dataset version:

@Roboflow, generate the dataset version using the preparation plan we just agreed. Check its status and return the version number, image counts, split counts, and applied settings.

Starting without added dataset augmentation makes the first experiment easier to interpret. It does not mean that the model trainer has no internal augmentation behavior. Roboflow’s augmentation guide (opens in a new tab) provides background for later experiments.

The above prompt will generate a dataset version. You'll see the following response:

Explore Model Architectures and Checkpoints

For the first run, use RF-DETR Small as a concrete baseline candidate. The goal is to learn how this dataset behaves before committing to a broader architecture search. A checkpoint supplies the model’s starting weights. For an initial experiment, request a compatible public pretrained checkpoint. Do not assume that weights from another architecture can be used interchangeably.

@Roboflow, review RF-DETR Small for [PROJECT_ID], version [VERSION_NUMBER]. Confirm compatibility and identify an available public pretrained checkpoint. Explain the tradeoffs for detecting small surface defects and list the training settings we can control through this workspace.

A useful architecture discussion connects model capacity to the images and the intended inspection system. Ask whether the defects are large enough to remain visible at the selected input resolution, what compute will be available for inference, and how much review time the application can tolerate. These questions help define the comparison.

For a later architecture exploration, consider Neural Architecture Search (opens in a new tab) if your workspace supports it and the dataset meets its requirements, including at least 15 validation images. It can explore speed–accuracy tradeoffs, but does not replace a representative dataset or a clear acceptance criterion.

Inspect and Understand the Available Training Settings

Before launching a run, inspect the settings available for the selected architecture and dataset version. This is a useful way to explore the training-and-evaluation skill in more depth. Depending on the model, the returned recipe may expose settings such as:

Try it using a prompt like this:

@Roboflow, describe the training recipe for RF-DETR Small on this version. Explain the returned defaults, valid settings, and suggested ranges in plain language. Check whether this workspace can submit a custom recipe. Do not start training.

The returned recipe will look similar to this:

Use the recipe returned for your exact model and dataset version. A valid range tells you what is accepted, it does not tell you which value will work best. Recipe inspection and custom-recipe submission have separate availability checks. If your workspace reports that custom submission is unavailable, continue with standard training controls. Keep advanced recipe changes as an optional extension.

For the baseline, use standard settings, a compatible checkpoint, and a planned maximum of 100 epochs. This is an experiment choice, not an established optimum for this dataset. Leave other settings at their supported defaults. Read Roboflow’s model-training documentation (opens in a new tab) for additional context.

Start Training and Monitor the Run

Before training, have ChatGPT restate the selected project, version, architecture, checkpoint, and epoch setting. This prevents ambiguity when several versions or experiments exist. Then launch the agreed run:

@Roboflow, start one standard RF-DETR Small training run on [PROJECT_ID], version [VERSION_NUMBER], using the compatible public pretrained checkpoint we selected and 100 epochs. Leave other settings at their supported defaults and do not submit a custom recipe. Return the training ID and the accepted configuration.

This will start the training run and you will see this:

Click on "View training in Roboflow" to see it. Record the returned training ID. When checking progress, refer to that run:

@Roboflow, check the current status of training [TRAINING_ID] in [PROJECT_ID], version [VERSION_NUMBER].

If training is still running, report its status and any available progress.

If training has finished, list the models produced and their exact model IDs.

You'll see the following response:

Do not interpret a long-running status check as a reason to launch another job. Training is asynchronous, return to the conversation to request another update. Roboflow distinguishes versions, trainings, and models (opens in a new tab). A dataset version can have multiple training runs, and a run can produce one or more models. Keeping these identifiers together makes later comparisons traceable.

If you need to interrupt training, ask about the distinction between stopping and canceling a job (opens in a new tab). Early stopping can retain training progress as a model; cancellation discards the run’s artifacts. Compute already used still matters for credit consumption.

Read and Visualize the Evaluation Results

After training completes, retrieve the model’s available evaluation. Detailed evaluation may still be processing or may depend on workspace access.

@Roboflow, find the completed evaluation for [MODEL_ID]. Use the validation split for this analysis. Show mAP@50, mAP@50:95, and the available per-class precision, recall, and F1 for scratch and paint-damage. Identify the evaluation ID, split, and confidence threshold used for threshold-dependent metrics. If results are unavailable, explain the returned status.

The response will look like this:

Read the metrics in the context of the inspection task:

An overall score is only the beginning. If scratch recall is lower than paint-damage recall, investigate the scratch examples before concluding that the entire model needs replacement. Also check how many examples support each result. A few additional errors can substantially change a metric on a small evaluation set.

The evaluation documentation (opens in a new tab) explains the available analysis views. Record the results returned for your model and evaluation ID. Use a chart when it makes class differences easier to assess:

@Roboflow, create a bar chart of validation mAP@50:95 for scratch and paint-damage using the retrieved results. Label the model, dataset version, and split. Also display the available confusion matrix with clearly labeled actual and predicted axes. If a result is missing, identify it explicitly.

Here's the output:

The chart should make the evidence easier to read. Keep the underlying values beside it so the result remains traceable.

Explore Confidence Thresholds

A confidence threshold determines which predictions are retained. It does not retrain the model, and a confidence value is not the same as overall model accuracy. In the proposed inspection station, lowering the threshold may help surface more defects while increasing the number of regions an inspector must review. Explore that tradeoff using validation data. Use following prompt:

@Roboflow, retrieve the confidence sweep for this model’s evaluation. Use validation results to compare three candidate operating thresholds: one emphasizing recall, one near the best F1, and one emphasizing precision. Show the available overall and per-class results. Explain the likely effect on missed defects and review workload.

This will give you this:

Treat the candidates as operating choices to investigate. The threshold with the best F1 may not fit a workflow where missing a defect carries a different cost from reviewing a false alarm. If false-positive counts are available, relate them to the number of inspected validation images. That provides a more concrete discussion than a percentage alone. Choose a provisional threshold using validation evidence, then keep the test set untouched until the model and operating point are finalized.

Diagnose Model Weaknesses Using the Skill

Metrics tell you where to look. Individual examples help explain what happened.

@Roboflow, use the roboflow-training-and-evaluation skill to retrieve the validation confusion matrix and per-image error results at the selected threshold. Identify examples containing missed scratches, false paint-damage detections, and class confusion where present. Show the corresponding images and available ground-truth annotations and predictions. If exact prediction coordinates are unavailable, state that rather than creating an approximate overlay.

The prompt will generate the following response:

Here is the validation example image that will be generated:

Read the confusion matrix with its actual and predicted axes clearly labeled. Then inspect the images behind the counts. For this application, useful hypotheses include:

These are hypotheses, not findings about the public dataset. Ask ChatGPT to distinguish three things in its response:

• what the evaluation reports,
• what image inspection supports, and
• what still needs testing.

Where available, vector analysis and automated recommendations can help identify patterns worth investigating. They should guide review, not replace inspection of the underlying examples.

Run a Controlled Training Comparison

Use the diagnosis to decide whether another training configuration is worth testing. Keep the dataset version fixed so that the comparison addresses a training choice. For example, if the available training results suggest a longer run may help, you could compare two epoch budgets:

This is an illustrative experiment plan, not a claim that 150 epochs will improve the model. Early stopping may end either run before its budget is reached. Record the actual completed epochs where available.

@Roboflow, use our validation findings to propose one supported training-setting change on the same dataset version. Explain the hypothesis, keep the other controllable settings fixed, and identify the validation metrics that will determine whether the change helped. Present the experiment plan before starting.

The above prompt will produce the following result:

After reviewing the plan, run:

@Roboflow, start the candidate run using the training configuration we agreed. Return the training ID and accepted settings so we can compare it with the baseline.

This will start the training run.

The candidate training run will be started in Roboflow.

Use the model-comparison capability (opens in a new tab) to inspect models from the same dataset version. Retrieve each candidate’s validation metrics for selection and check the split behind any summary score.

@Roboflow, compare the baseline and candidate using their exact model and evaluation IDs. Use validation metrics for model selection. Show the class-level results, confidence thresholds, completed training settings, and any available latency measurements with their measurement context. Identify missing or incomplete evaluations.

Here are the comparison results:

For wheel inspection, check whether a gain in overall mAP also helps the weaker defect class. A model that improves paint-damage detection while missing more scratches may not fit the inspection objective. Treat a small difference from a single pair of runs as provisional evidence. Training variation and the size of the validation set affect how confidently you can interpret it. Server-side inference timing also needs context, it is not the full time required for camera capture, image transfer, inference, and an inspection decision.

Improve the Dataset and Retrain

Choose the next experiment from the strongest evidence. Suppose validation review shows that faint scratches on reflective surfaces are often missed. A reasonable hypothesis is that the training set does not represent those conditions well enough. You could collect additional images under the relevant lighting, review their annotations, and add them to training. Keep the held-out validation and test images separate.

@Roboflow, summarize the strongest supported error pattern and propose one follow-up experiment. State the hypothesis, the dataset or training change, the settings we will keep fixed, and the validation result that would support the hypothesis. Identify any additional images or annotation review needed before we can execute it.

For a data-focused experiment, hold the architecture, starting checkpoint, and main training settings constant. This makes the effect of the dataset change easier to assess.

For an augmentation experiment, choose transformations that plausibly occur at the inspection station. Modest brightness variation may be useful if lighting varies. Strong hue changes could obscure the appearance cues used to recognize paint damage.

Keep generated training variants out of the validation and test sets. If you discover incorrect held-out labels, correct them transparently and reevaluate both models against the same corrected reference. After the change is prepared and reviewed:

@Roboflow, create the next dataset version with the agreed change. Preserve the held-out image membership and record any label corrections. Train the same baseline configuration so we can compare the experiments fairly.

The built-in model comparison groups models from the same dataset version. After changing the dataset, explicitly check that the held-out images and reference annotations remain comparable.

Create a Final Training and Evaluation Report

Once you have selected the model and threshold, evaluate them on the held-out test set. Avoid repeatedly changing the model in response to that final test result; continued iteration needs a fresh plan for unbiased evaluation. Ask ChatGPT to produce a report that another engineer can follow:

@Roboflow, evaluate our selected model at the agreed operating threshold on the held-out test set, using the available completed evaluation. Then prepare a training report containing the dataset source and license, project and version identifiers, training configuration, model IDs, validation comparison, final test results, important failure examples, and limitations. Separate measured results from proposed next steps. Mark any unavailable result explicitly.

The above prompt will generate a complete report like this:

The report should explain what changed and what the evidence supports. “Added reviewed examples of faint scratches and measured the effect on scratch recall” is a stronger account than “optimized the model,” provided the recorded results support it.

Keep the final configuration, model ID, evaluation ID, and selected threshold together. They identify the model and operating point the report actually assessed. Any production pilot should use representative manufacturing images and acceptance criteria agreed by the inspection team.

Make Training Decisions From Evidence

Computer vision training becomes more useful when each decision is tied to evidence. In alloy-wheel inspection, that means understanding the labeled defects, examining missed scratches and false alarms, and measuring whether a change improves the inspection task.

The Roboflow plugin in ChatGPT brings dataset exploration, training, and evaluation into one conversation. The roboflow-training-and-evaluation skill helps connect those activities so that model selection, threshold choices, and retraining follow a clear line of reasoning.

Start with a relevant dataset, establish a baseline, and use the evaluation to choose your next experiment. For a broader introduction to the workflow, explore the Roboflow Model Training guide (opens in a new tab).