5 papers
ObjectForesight: Predicting Future 3D Object Trajectories from Human Videos
Rustin Soraki, Homanga Bharadhwaj, Ali Farhadi +1
Humans can effortlessly anticipate how objects might move or change through interaction--imagining a cup being lifted, a knife slicing, or a lid being closed. We aim to endow compu…
Grounding Multimodal LLMs to Embodied Agents that Ask for Help with Reinforcement Learning
Ram Ramrakhya, Matthew Chang, Xavier Puig +3
Embodied agents operating in household environments must interpret ambiguous and under-specified human instructions. A capable household robot should recognize ambiguity and ask re…
ADAPT: Actively Discovering and Adapting to Preferences for any Task
Maithili Patel, Xavier Puig, Ruta Desai +4
Assistive agents should be able to perform under-specified long-horizon tasks while respecting user preferences. We introduce Actively Discovering and Adapting to Preferences for a…
From an Image to a Scene: Learning to Imagine the World from a Million 360 Videos
Matthew Wallingford, Anand Bhattad, Aditya Kusupati +7
Three-dimensional (3D) understanding of objects and scenes play a key role in humans' ability to interact with the world and has been an active area of research in computer vision,…
PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks
Matthew Chang, Gunjan Chhablani, Alexander Clegg +17
We present a benchmark for Planning And Reasoning Tasks in humaN-Robot collaboration (PARTNR) designed to study human-robot coordination in household activities. PARTNR tasks exhib…