438 citations · 680 across the 15 of their papers we have counts for
11 papers · 1 filter
Moving Forward by Moving Backward: Embedding Action Impact over Action Semantics
Kuo-Hao Zeng, Luca Weihs, Roozbeh Mottaghi +1
A common assumption when training embodied agents is that the impact of taking an action is stable; for instance, executing the "move ahead" action will always move the agent forwa…
FastFill: Efficient Compatible Model Update
Florian Jaeckle, Fartash Faghri, Ali Farhadi +2
In many retrieval systems the original high dimensional data (e.g., images) is mapped to a lower dimensional feature through a learned embedding model. The task of retrieving the m…
Patching open-vocabulary models by interpolating weights
Gabriel Ilharco, Mitchell Wortsman, Samir Yitzhak Gadre +5
Open-vocabulary models like CLIP achieve high accuracy across many image classification tasks. However, there are still settings where their zero-shot performance is far from optim…
Break and Make: Interactive Structural Understanding Using LEGO Bricks
Aaron Walsman, Muru Zhang, Klemen Kotar +3
Visual understanding of geometric structures with complex spatial relationships is a fundamental component of human intelligence. As children, we learn how to reason about structur…
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound
Rowan Zellers, Jiasen Lu, Ximing Lu +7
As humans, we navigate a multimodal world, building a holistic understanding from all our senses. We introduce MERLOT Reserve, a model that represents videos jointly over time -- t…
Forward Compatible Training for Large-Scale Embedding Retrieval Systems
Vivek Ramanujan, Pavan Kumar Anasosalu Vasu, Ali Farhadi +2
In visual retrieval systems, updating the embedding model requires recomputing features for every piece of data. This expensive process is referred to as backfilling. Recently, the…