01 · Case study · Sports analytics
Automated basketball stats from ordinary game video
A youth, amateur and semi-professional basketball analytics platform tagged footage by hand, and its analysts fell behind every tournament weekend. Brainy Neurals built an automated video pipeline that detects, tracks and scores every play from uploaded footage. Seven event types, per-player stats, clips and an annotated video return from every upload, with no hand tagging.
02 · Quick answers
At a glance
What problem did this solve?
Manual tagging of basketball games was slow and inconsistent at a youth and amateur analytics platform. Its analysts couldn’t keep up with tournament volume.
What did Brainy Neurals build?
Brainy Neurals built an automated basketball video pipeline that detects and tracks players and the ball. It recognizes seven event types and exports per-player stats, clips and annotated video.
What changed after it went live?
After go-live, every uploaded game returns a stat sheet, event clips and an annotated video without an analyst tagging it. The client reports lower turnaround and cost per game.
Who else could use this?
Automated event tagging suits any organization that still logs what happened on video by hand. That includes other sports, retail stores, warehouses, construction sites and broadcasters.
Engagement facts
| Industry | Sports analytics |
|---|---|
| Sub-vertical | Youth and amateur basketball |
| Client | A basketball analytics platform |
| Engagement | End-to-end pipeline build |
| Timeline | Not disclosed |
| Capabilities | Detection, tracking, events, chat |
| Delivery model | Cloud pipeline, API delivery |
03 · The problem
Why did manual game tagging keep failing?
A basketball analytics platform for youth, amateur and semi-professional teams promised every coach a breakdown of every game. Each breakdown depended on an analyst watching the footage and tagging events by hand. The platform asked Brainy Neurals for a computer vision development program that could take tagging off human hands.
-
1
A single game took longer to tag by hand than to play, and a tournament day produced dozens.
-
2
Two analysts tagging the same game disagreed on assists and rebounds, so the stat sheet changed with the person.
-
3
Tournament weekends piled up untagged games faster than any analyst roster could clear them.
-
4
Coaches received the breakdown too late to use it, often after the next game had already been played.
Published performance analysis gets two observers to agree by defining every action in writing first (Francis et al., 2019). Hand tagging at tournament pace skips that step.
04 · The alternatives
What do teams usually try first?
Most programs try one of three things first, and each works up to a point.
| Approach | What it gets right | Where it stops | Who it still suits |
|---|---|---|---|
| Analyst tagging by hand | Judgment on gray-area calls | Hours per game, and it does not scale | Elite programs with budgets |
| Live stats app, one scorer | Cheap and instant box score | No clips, one pair of eyes | Leagues that want a box score |
| Auto-tracking camera | Follows play, cuts highlights | Stats stop at highlights, hardware lock-in | Clubs that mainly want broadcast |
| Software tagging of ordinary footage (this build) | Events, stats and clips from any recording | Needs varied footage and tuning | Platforms processing games at volume |
05 · The design
How we designed the tagging pipeline
Brainy Neurals built an automated video intelligence pipeline for a youth and amateur basketball analytics platform.
The pipeline reads each game frame by frame, holds one identity per player from tip-off to buzzer, and separates the two teams. Our video analytics work on crowded scenes is where the tracking came from. Event logic then reads the play from how the ball, the net and the players move together.
Decision 01
Infer events from geometry
A make is the ball passing through the net. Rules like that can be read and tuned with a coach in the room.
We rejected one model that predicts events end to end because it needs far more labeled footage and fails silently. A coach can argue with a rule, and nobody can argue with a black box.
Decision 02
Queue games, do not stream them
A game is uploaded as a file, queued, processed and delivered back to the platform. Live courtside processing would have meant hardware in every gym. Because the volume lives in tournament peaks, a queue absorbs a peak where a live system falls over.
A coach can argue with a rule, and nobody can argue with a black box.
06 · The stack
The technology stack we used
Everything in this stack answers one constraint. Hundreds of thousands of games a year, each a long video, on shared GPUs, at youth league prices.
So we kept the models small, the pipeline stream-based and the service layer light. A generative AI development layer sits on top, so a coach can question the sheet in plain English.
| Layer | What we used | Why | What we ruled out |
|---|---|---|---|
| 01Detection | Compact object detectors per target | Each target fails differently | One big model |
| 02Tracking | Jersey-aware tracker, persistent IDs | Same player, whole game | Appearance-only matching |
| 03Events and scoring | Rules over geometry and zones | A coach can argue with it | One classifier per event |
| 04Inference | Shared GPU inference, reduced precision | Cheap per game at volume | Frame by frame on CPU |
| 05Service layer | A light async service layer | The platform polls for progress | A workflow engine |
| 06Storage | Cloud storage and portable exports | Simple to move and to read | Managed database, for now |
| 07Analytics chat | A language layer tied to the sheet | Every answer traces to a row | Ungrounded chat |
07 · One game, end to end
What happens after a coach hits upload?
-
01
Upload
A coach uploads the game recording, which lands in cloud storage and joins the queue.
-
02
Detect and track
The pipeline reads the footage frame by frame and finds the players, the ball and the net. It then holds one identity per player for the whole game and separates the two teams.
-
03
Read the play
Event logic watches ball, net and players together and calls possessions, shots, makes, rebounds, assists, steals and blocks. A court map read from the footage places each shot in its zone, so a make scores one, two or three points.
-
04
Build outputs
Stats compile per player, timestamped to the event, and export as CSV. Clips cut around each key event, and an annotated video renders with scoreboard, trails and labels.
-
05
Deliver
The run completes, the platform is told it is done, and the outputs come back as links a coach can open. The coach reads the sheet or asks the assistant a question.
Want this walked through for your setup?
No pitch. If it is not a fit, you will know in five minutes.
08 · The hard part
What went wrong on the court?
-
The ball vanished at the moments that mattered. Small, fast and usually hidden behind the people chasing it, it went missing at release and at the rim.
-
Player identities swapped. Two players in the same kit crossed, and the tracker handed each the other’s number for a whole quarter. Jersey numbers did not rescue it, because a number is legible in few frames (Balaji et al., 2023).
-
Team colors blurred under gym lights. White against light gray was a coin toss, and referees and the bench kept joining a team.
-
Every gym was a different court. US high school arcs sit at 19.75 feet (6.02 meters), FIBA arcs at 22.15 feet (6.75 meters). A zone map built on one floor put threes in the wrong place on the next.
Tracking benchmarks blame the same two things, fast uneven motion and look-alike players (Cui et al., 2023). Most teams need specialist engineers at this point rather than another analyst.
09 · The engineering
How we solved each of them
FIX 01
The vanishing ball
So we stopped demanding the ball in every frame and started trusting the net. A make became the ball passing through the net, a steadier signal than a blur leaving a hand.
FIX 02
Swapped identities
We made identity a running judgment instead of a single snapshot. Evidence gathered across the whole game outweighs any one bad frame. Team color narrows the choice, and a player can’t be in two places at once.
FIX 03
Blurred teams
We let each game define its own two teams rather than matching a fixed color list. The court boundary keeps referees and the bench out of both.
FIX 04
A different court every night
The pipeline reads the lines in each recording and maps the zones from them. The three-point arc is wherever that floor says it is.
None of the four fixes needed a new model, and that is how we work on every build.
10 · After go-live
What changed after go-live
| Measure | Before | After |
|---|---|---|
| Tagging | An analyst, one possession at a time | Automatic, for every game uploaded |
| Stat sheet | Box score typed by hand | Per player, timestamped, exported as CSV |
| Clips | Cut by hand for a few moments | Cut around every key event |
| Delivery | When an analyst got to it | Links a coach opens, on completion |
| Questions | Wait for the analyst | Ask the chat assistant |
We have not published a turnaround or cost figure for this build. The client reports both fell, and a number we have not measured is a number we will not print.
Day to day, the analyst’s role moved from tagging every possession to checking the output. The platform can now take a tournament’s full slate instead of the handful its analysts could reach. And the stat sheet now reaches clubs that could never have paid for an analyst.
What do you still tag by hand?
Tell us what your people still log from video, screens or paper. If a pipeline like this one fits, you will hear how. If it does not, you will hear that too.
No sales sequence. One reply, from the person who would architect it.
11 · In production
What is running today
The pipeline runs inside the client’s platform today, behind its own upload and status endpoints. Every uploaded game goes through the same chain and comes back as stats, event clips and an annotated video.
Brainy Neurals built the pipeline for tournament-scale volume rather than a handful of showcase games. Capacity grows by adding workers, not by adding analysts.
This is the kind of performance analytics that used to stop at elite programs. Now it reaches academies and club teams.
12 · Lessons
What we would do differently
Trust the net from day one.
We spent weeks chasing per-frame ball detection before accepting that the net is the steadier witness to a make.
Read the court from the footage before training anything.
But a fixed zone template cost us a rebuild after the second venue.
Agree the stat definitions with coaches before tagging a single game.
An assist means different things to different coaches, and a model cannot settle that argument.
Build the queue before the models are good.
The pipeline shape, not the model, decided whether the platform could survive a tournament weekend.
An assist means different things to different coaches, and a model cannot settle that argument.
13 · The pattern
Where else this pattern fits
Automated video event tagging is software that logs what happened in footage by tracking the people and objects in it. It fits wherever people still watch recordings and write down what happened.
| Industry | The equivalent problem | What changes in the build |
|---|---|---|
| Other sports | Soccer, hockey, volleyball, same backlog | New event rules and a different field map |
| Retail | Shopper paths and shelf interactions logged by hand | Shelf zones for court zones, dwell for possession |
| Warehouse operations | Picks, drops and near misses go unrecorded | Forklifts and pallets replace players and ball |
| Construction | Safety breaches logged by a walking supervisor | Exclusion areas for zones, breaches for events |
| Broadcast | Editors cutting highlights by hand | Same clip engine, new event set |
Porting it takes new event rules, a new map of the space, and footage to tune on.
14 · Repeating it
What does it take to repeat this?
01How does AI basketball stat tracking work?
Automated basketball stat tracking starts with detectors that find players, ball, net and jersey numbers in every frame. A tracker follows each player, and event logic calls seven event types from how ball, net and players move. Court zones then assign the points.
02How accurate is AI basketball stat tracking?
Brainy Neurals does not publish accuracy figures for client systems. Every event on the sheet carries a timestamp, so a coach can check any call against its clip in seconds. The client’s analysts now review output instead of tagging from scratch.
03Do I need special cameras or sensors?
No sensors and no special cameras. The pipeline was built for ordinary game recordings, and court zones come from segmenting the footage itself. Footage that keeps the whole court in frame helps most.
04Does AI stat tracking work for youth and high school games?
Yes. Youth footage is harder than broadcast, with small gyms, mixed lighting and players of every size. Players are identified by jersey number and team color, not by face, which matters when players are minors.
05How long does it take to build automated game tagging?
Long enough to watch your own footage fail, which can’t be skipped. Brainy Neurals starts with a proof of concept on your recordings, then hardens the pipeline around what broke. Event set and footage variety decide the calendar, so we scope it on a scoping call.
06How much does a custom sports video analytics system cost?
The cost of a custom sports video analytics build turns on event set, footage variety and annual game volume. We don’t publish rates or price from a form. A scoped proof of concept on your footage gets a fixed price before any code is written.
Your analysts could be checking output instead of tagging it.
15 · The services
The services this was built from
The detectors for players, ball, net, jersey numbers and poses started here.
Video analytics →Tracking fast, look-alike people across long recordings is the same problem as a busy site camera.
Generative AI development →The chat layer that answers a coach’s question straight from the stat sheet.
RAG development →Grounding a language model in your own data, so every answer cites a row.
Edge AI →Moving the same detectors onto a courtside device when a venue needs results at halftime.
AI in sports →Player tracking and performance analytics for leagues, academies and analytics platforms, from youth up.
Not there yet? Start with an AI proof of concept, AI consulting or an AI readiness assessment, or browse all industries.