Attention-Based Keyword Localisation in Speech using Visual Grounding

Kayode Olaleye, Herman Kamper

Visually grounded speech models learn from images paired with spoken captions. By tagging images with soft text labels using a trained visual classifier with a fixed vocabulary, pr…