978 citations · 7k across the 148 of their papers we have counts for
30 papers · 1 filter
Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation
Matthieu Futeral, Cordelia Schmid, Ivan Laptev +2
One of the major challenges of machine translation (MT) is ambiguity, which can in some cases be resolved by accompanying context such as images. However, recent work in multimodal…
Audiovisual Masked Autoencoders
Mariana-Iuliana Georgescu, Eduardo Fonseca, Radu Tudor Ionescu +3
Can we leverage the audiovisual information already present in video to improve self-supervised representation learning? To answer this question, we study various pretraining archi…
REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory
Ziniu Hu, Ahmet Iscen, Chen Sun +6
In this paper, we propose an end-to-end Retrieval-Augmented Visual Language Model (REVEAL) that learns to encode world knowledge into a large-scale memory, and to retrieve from it…
Location-Aware Self-Supervised Transformers for Semantic Segmentation
Mathilde Caron, Neil Houlsby, Cordelia Schmid
Pixel-level labels are particularly expensive to acquire. Hence, pretraining is a critical step to improve models on a task like semantic segmentation. However, prominent algorithm…
AVATAR submission to the Ego4D AV Transcription Challenge
Paul Hongsuck Seo, Arsha Nagrani, Cordelia Schmid
In this report, we describe our submission to the Ego4D AudioVisual (AV) Speech Transcription Challenge 2022. Our pipeline is based on AVATAR, a state of the art encoder-decoder mo…
Language Conditioned Spatial Relation Reasoning for 3D Object Grounding
Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi +2
Localizing objects in 3D scenes based on natural language requires understanding and reasoning about spatial relations. In particular, it is often crucial to distinguish similar ob…