The next iteration of The Master Group’s QC process: take the curated dataset our field inspections generated and train a detection transformer to flag product damage automatically — pushing inspection overhead toward zero and making results consistent across every centre, no matter who’s doing the looking. Snap a photo of a rooftop HVAC unit; the model flags damage on the spot. The hard part wasn’t the model — it was wringing signal from a sparse, noisy, real-world dataset, so I built an active-learning labeling loop that ranks every correction by model disagreement and ran the levers to their limits.
I directed AI agents to build the training pipeline, the active-learning loop, and the serving app — then used my own judgment to run the experiments and read the results. That judgment is what caught a data-leakage mirage the metrics were happy to ship. Directing AI gets you a working model fast; knowing what the numbers actually mean is what makes it trustworthy.
This is the field flow: the phone captures a unit, the model runs sliced inference, and damage regions come back with confidence scores. Cycle through sample units and watch the detector work.
Serving uses a high-recall union of full-image + sliced inference — small damage on a big unit is easy to miss at a single scale, so the model looks at both.
With a tiny, hard dataset, which images you label matters more than how many. The review queue is ordered by model-vs-label disagreement, so the highest-value corrections come first and clean agreements need only a glance. Deleting a false positive is captured as a hard negative that the next model is forced to learn from.
↑ Where does this data come from? A company-wide PowerApps / Power Automate / SharePoint QC platform I built — it captures the field inspections and cleans them into exactly the label pool this model trains on.
I systematically exhausted the improvement levers — data volume, input resolution, and model capacity — across 14 generations. Recall plateaued near 0.50 on all three, which told me something important: the ceiling was the label/task itself, not the model.
To test that, I probed a binary damage-gate paradigm and caught a data-leakage mirage — a "0.999" score that was really the model shortcutting on image source, not damage. Blur the batteries, and the honest same-source signal was AUROC ~0.61. Knowing what your metric is actually measuring is the whole job.