Leveraging multimodal explanatory annotations for video interpretation with Modality Specific Dataset

Elisa Ancarani, Julie Tores, Lucile Sassatelli +3

We examine the impact of concept-informed supervision on multimodal video interpretation models using MOByGaze, a dataset containing human-annotated explanatory concepts. We introd…