What Do Audio-Visual Synchronization Metrics Actually Measure?

Abstract

Automatic AV-sync metrics are widely used to rank and train audio-visual generators, but they are rarely audited as measurement instruments. We jointly audit AV-Align, ImageBind AV-relevance, JavisScore, and Synchformer/DeSync under a common reliability protocol: controlled-distortion monotonicity, preprocessing sensitivity, rank uncertainty, cross-metric agreement, PEAVS-proxy agreement, and learned fusion. The result is an axis split, not a single winner: Synchformer/DeSync is the strongest temporal-offset tracker ($\tau =0.84$), ImageBind/JavisScore better match the PEAVS human-aligned proxy ($\tau =0.20$) and content-disruption families, and AV-Align is the weakest standalone metric. The metrics mutually disagree (Krippendorff $\alpha =0.066$), and neither linear nor simple $k$-NN fusion improves PEAVS agreement over the best individual metric. We recommend reporting AV-sync as a Reliability Card (metric-family breakdowns with confidence intervals) rather than a single bare synchronization score.

Accepted at the ECCV 2026 Workshop on Generative AI for Audio-Visual Content Creation (Gen4AVC), poster presentation; non-archival workshop. 7 pages (4-page main text + references + 2-page appendix), 3 figures, 8 tables. Project page: https://jaishrm07.github.io/avsync-reliability-card/