Vision-Speech Models: Teaching Speech Models to Converse about Images

Amélie Royer, Moritz Böhle, Gabriel de Marmiesse +4

The recent successes of Vision-Language models raise the question of how to equivalently imbue a pretrained speech model with vision understanding, an important milestone towards b…