MVTrack: Ultrafast Appearance-Free Moving Object Tracking from Compressed Bitstreams

Abstract

Deploying modern video trackers at scale is bottlenecked by the computational cost of RGB-based object detectors. To this end, we present MVTrack, an ultrafast tracker for moving objects that operates directly on H.264 bitstreams. MVTrack combines MVDet, a lightweight detector for motion vector fields, with MVLink, a minimalist kinematic association module. On VIRAT, MVTrack outperforms YOLO26n while using 60$\times$ fewer parameters, requiring 40$\times$ fewer FLOPs, and reducing CPU latency by 8.6$\times$. These results demonstrate that compressed video data alone can enable accurate and scalable surveillance tracking, thereby bypassing the need for pixel reconstruction.

This paper has been accepted to the 2nd workshop on Low-Level Vision Frontiers with Generative AI, Preference Optimization, Agentic Systems and World Models (LoViF) at ECCV2026