206 citations · 334 across the 15 of their papers we have counts for
10 papers · 1 filter
Behavioral Analysis of Vision-and-Language Navigation Agents
Zijiao Yang, Arjun Majumdar, Stefan Lee
To be successful, Vision-and-Language Navigation (VLN) agents must be able to ground instructions to actions based on their surroundings. In this work, we develop a methodology to…
OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNav
Karmesh Yadav, Arjun Majumdar, Ram Ramrakhya +5
We present a single neural network architecture composed of task-agnostic components (ViTs, convolutions, and LSTMs) that achieves state-of-art results on both the ImageNav ("go to…
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?
Arjun Majumdar, Karmesh Yadav, Sergio Arnaud +12
We present the largest and most comprehensive empirical study of pre-trained visual representations (PVRs) or visual 'foundation models' for Embodied AI. First, we curate CortexBen…
ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings
Arjun Majumdar, Gunjan Aggarwal, Bhavika Devnani +2
We present a scalable approach for learning open-world object-goal navigation (ObjectNav) -- the task of asking a virtual robot (agent) to find any instance of an object in an unex…
Offline Visual Representation Learning for Embodied Navigation
Karmesh Yadav, Ram Ramrakhya, Arjun Majumdar +5
How should we learn visual representations for embodied agents that must see and move? The status quo is tabula rasa in vivo, i.e. learning visual representations from scratch whil…
SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigation
Abhinav Moudgil, Arjun Majumdar, Harsh Agrawal +2
Natural language instructions for visual navigation often use scene descriptions (e.g., "bedroom") and object references (e.g., "green chairs") to provide a breadcrumb trail to a g…
Sim-to-Real Transfer for Vision-and-Language Navigation
Peter Anderson, Ayush Shrivastava, Joanne Truong +4
We study the challenging problem of releasing a robot in a previously unseen environment, and having it follow unconstrained natural language navigation instructions. Recent work o…
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web
Arjun Majumdar, Ayush Shrivastava, Stefan Lee +3
Following a navigation instruction such as 'Walk down the stairs and stop at the brown sofa' requires embodied AI agents to ground scene elements referenced via language (e.g. 'sta…
Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments
Jacob Krantz, Erik Wijmans, Arjun Majumdar +2
We develop a language-guided navigation task set in a continuous 3D environment where agents must execute low-level actions to follow natural language navigation directions. By bei…
Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning
David Mascharka, Philip Tran, Ryan Soklaski +1
Visual question answering requires high-order reasoning about an image, which is a fundamental capability needed by machine systems to follow complex directives. Recently, modular…