FERA: A Pose-Based Framework for Rule-Grounded Multimedia Decision Support with a Foil Fencing Case Study

Abstract

Multimedia decision support requires more than recognition; it requires explicit state estimates that can be checked against rules, audited by humans, and consumed by downstream decision logic. We present the FEncing Referee Assistant (FERA), a pose-based framework for this setting, and study it through foil fencing, where decisions depend on fast bilateral motion and right-of-way rules. The framework separates canonical participant tracking, kinematic tokenization, calibrated temporal perception, a compact structured decision layer, and an explanation-oriented retrieval interface. We also release an audited benchmark with adjudicated labels and fixed folds for reproducible evaluation. Under a shared protocol, a lightweight lifted-depth sidecar strengthens the best graph-based perception model, while a compact structured classifier on the fixed two-dimensional token stream reaches 0.624 accuracy and a 0.632 macro-averaged F1 score on the final Left / Right / None decision. The case study supports a broader design lesson: keep the boundary between perception and rule application explicit, preserve uncertainty, and choose the perception front end according to the downstream operating point.

Changed which model architectures were tested in the pipeline, introduced deeper end to end analysis, increased performance in move detection