Project
FutBot Tracker
FutBot Tracker helps robot-soccer video analysts inspect robot and ball tracks, possession states, annotated footage, and movement heatmaps.
- Period
- June 2026
- Status
- Competition entry
- Stack
- Python 3.12SAM 3UltralyticsOpenCVNumPyPyYAML
The pipeline
Two seconds of robot soccer. Every frame accounted for.
- 1 · Detection0 objects this frameOpenCV boxes refined by SAM 3 masks
- 2 · Confidenceno objects this frameper-object detection score
- 3 · Tracker0 robots · ball lostclass-aware, velocity-predicted
- 4 · Possessionunknown · no owner yetobserved · inferred · free · unknown
What it does
Robot-soccer video in, robots, ball and possession out.
- Detects and tracks every robot and the ball, frame by frame.
- Says who has the ball, and how sure it is.
- Exports annotated video, heatmaps and data files.
01Context
The problem,and my part in it.
The problem
Robot-soccer footage combines small moving objects, dark robot bodies, colored team markers, an orange ball, and intermittent occlusion. A useful output needs both object locations and continuity: a sequence of boxes alone does not tell a viewer which robot is moving or which robot is nearest to the ball.
What I did
I designed and built the whole pipeline, from raw video to tracked robots, ball and possession, and submitted the entry.
- Ran fast colour-and-shape proposals first and let SAM 3 refine only those boxes, which keeps the heavy model cheap.
- Wrote a custom tracker that keeps each robot's team and identity through occlusions.
- Built a possession estimator that says when it's guessing: observed, inferred or unknown.
- Delivered annotated video, heatmaps, data exports and automated tests.
02Decisions
How it was built,and why.
Candidate boxes make the boundary between domain rules and mask refinement inspectable. Full-frame independent model discovery is an alternative architecture; this implementation refines regions selected by color and geometry.
03Occlusion
When the ball hides behind a robot,the tracker says so.
The ball is visible: possession goes to the nearest robot within reach.
The ball is hidden: the last owner is carried forward, marked as an estimate.
No owner yet: it says so instead of picking one.
04Next
Where it goesfrom here.
- 01
Make it run on new footage without retuning the color ranges by hand.
- 02
Evaluate on longer matches and several camera setups.
- 03
Compare the custom tracker with established tracking methods.