Oliver Hazard

COURT

San Francisco AI referee

The AI referee

I'm building the AI referee. It should see what happened on the court, judge the play and explain the call so officiating becomes more transparent, consistent and trustworthy. Seeing is not enough; judgment remains the harder frontier.

  • 0.938 AUROC
  • 11,900-clip corpus
  • one broadcast camera

Demo

The demo follows the whole pipeline: one broadcast camera, players tracked and identified, contact isolated, a call drafted with its evidence, and an expert ruling on it.
The COURT pipeline from game film through candidate moments and grounded evidence to an expert ruling Game film Candidate moments Grounded evidence Expert ruling
COURT studio showing a graded moment
COURT studio, graded moment

What I built

Detection. I trained an E2E-Spot RegNetY-008 + GSM + GRU foul-spotting model on a proprietary 11,900-clip NBA corpus spanning three seasons: 2,548 fouls, 1,975 human-annotated contact frames and 9,215 negatives across 24 adversarial subcategories.

Grounded judgment. I built a perception dossier that gives a frontier model verified evidence of the players, contact and movement instead of asking it to guess from pixels.

Perception infrastructure. I built the modular tracking, team, jersey and court stack underneath it. It has 100 passing tests. Label-free QA across 20 clips carries 89.6% of detections into tracks with four jersey conflicts.

I attacked my own benchmark

My first model saturated the first benchmark: 96.5% F1, with 98.6% recall and 94.5% precision, on 358 held-out clips.

So I made the test adversarial. I mined blocked shots, contested misses, putbacks and transition collisions until hard negatives outnumbered fouls 3.6:1. On 301 fouls and 1,070 hard negatives, the model still separates the classes at 0.938 AUROC. At the recall-priority threshold it reaches 96.3% recall.

Over Christmas I ran three dataset-attack trainings in four days. Each targeted a named failure mode. Then I took the model on purpose into full broadcasts it had never seen, and mined its false positives straight back into the corpus as new negative classes. That is the flywheel working.

The v5 checkpoint also produced the demo artifact above: 18 detections at threshold 0.7 over a 30-minute broadcast segment, from real inference.

The COURT adversarial test: 301 fouls and 1,070 look-alikes across 1,371 clips, with operating-point results The test · 1,371 clips 301 fouls 1,070 look-alikes AUROC 0.938 Recall 87.0% Precision 68.4% False + 11.3%
Founder-produced test of curated clips. The chart shows the best-F1 operating point; the recall-priority threshold reaches 96.3% recall.

What I found

A frontier vision model agreed with the on-court call on 74% of 50 clips with broadcast audio, then 20% of 40 clips without it. It was listening to the announcer, not watching the play. I found the shortcut by ablation in my own setup. I now run audio off.

Agreement with the on-court call was 74 percent with audio on in a run of 50 clips and 20 percent with audio off in a run of 40 clips Audio on · n=50 74% Audio off · n=40 20%
Saved trials exposed the audio shortcut. Audio is now off in my evaluation setup.

On my founder-labeled benchmark of nine plays, ungrounded models called a foul on about 96% of clean plays. They fabricated jersey numbers in 92% of their identity claims and reported about 95% confidence when wrong. Giving them the verified perception dossier cut identity fabrication to near zero.

Why evaluation comes first

A machine cannot learn to judge without expert judgment to learn from. Referee evaluation is where experts already produce it: a graded call, bound to video, one at a time. Evaluation is not a smaller ambition; it is the path to the AI referee.

I built the benchmark, made it adversarial against myself and found the field's blind spot by ablation. The point is not a model that sounds certain. It is a system that shows its evidence and puts the ruling in expert hands.

Read the methods and limits for the full evaluation record.