22k citations · 22.3k across the 11 of their papers we have counts for
12 papers
Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini
Madhuri Shanbhogue, Zhe Li, Shanfeng Zhang +86
We introduce Gemini Embedding 2, a native multimodal embedding model that allows embedding video, audio, image, and text modalities in a unified representation space. We leverage t…
Benchmarking LLMs on the Massive Sound Embedding Benchmark (MSEB)
Cyril Allauzen, Tom Bagby, Georg Heigold +2
The Massive Sound Embedding Benchmark (MSEB) has emerged as a standard for evaluating the functional breadth of audio models. While initial baselines focused on specialized encoder…
Massive Sound Embedding Benchmark (MSEB)
Georg Heigold, Ehsan Variani, Tom Bagby +4
Audio is a critical component of multimodal perception, and any truly intelligent system must demonstrate a wide range of auditory capabilities. These capabilities include transcri…
Video OWL-ViT: Temporally-consistent open-world localization in video
Georg Heigold, Matthias Minderer, Alexey Gritsenko +5
We present an architecture and a training recipe that adapts pre-trained open-world image models to localization in videos. Understanding the open visual world (without being const…
Conditional Object-Centric Learning from Video
Thomas Kipf, Gamaleldin F. Elsayed, Aravindh Mahendran +6
Object-centric representations are a promising path toward more systematic generalization by providing flexible abstractions upon which compositional world models can be built. Rec…
ViViT: A Video Vision Transformer
Anurag Arnab, Mostafa Dehghani, Georg Heigold +3
We present pure-transformer based models for video classification, drawing upon the recent success of such models in image classification. Our model extracts spatio-temporal tokens…