6.2k citations · 14.2k across the 134 of their papers we have counts for
20 papers · 1 filter
How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget
Erik Wijmans, Irfan Essa, Dhruv Batra
PointGoal navigation has seen significant recent interest and progress, spurred on by the Habitat platform and associated challenge. In this paper, we study PointGoal navigation un…
SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency
Sameer Dharur, Purva Tendulkar, Dhruv Batra +2
Recent research in Visual Question Answering (VQA) has revealed state-of-the-art models to be inconsistent in their understanding of the world -- they answer seemingly difficult qu…
Sim-to-Real Transfer for Vision-and-Language Navigation
Peter Anderson, Ayush Shrivastava, Joanne Truong +4
We study the challenging problem of releasing a robot in a previously unseen environment, and having it follow unconstrained natural language navigation instructions. Recent work o…
Rearrangement: A Challenge for Embodied AI
Dhruv Batra, Angel X. Chang, Sonia Chernova +9
We describe a framework for research and evaluation in Embodied AI. Our proposal is based on a canonical task: Rearrangement. A standard task can focus the development of new techn…
Bi-directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents
Joanne Truong, Sonia Chernova, Dhruv Batra
Deep reinforcement learning models are notoriously data hungry, yet real-world data is expensive and time consuming to obtain. The solution that many have turned to is to use simul…
Learning Navigation Skills for Legged Robots with Learned Robot Embeddings
Joanne Truong, Denis Yarats, Tianyu Li +4
Recent work has shown results on learning navigation policies for idealized cylinder agents in simulation and transferring them to real wheeled robots. Deploying such navigation po…
Where Are You? Localization from Embodied Dialog
Meera Hahn, Jacob Krantz, Dhruv Batra +4
We present Where Are You? (WAY), a dataset of ~6k dialogs in which two humans -- an Observer and a Locator -- complete a cooperative localization task. The Observer is spawned at r…
Numerical scheme for the far-out-of-equilibrium time-dependent Boltzmann collision operator: 1D second-degree momentum discretisation and adaptive time stepping
Indrajit Wadgaonkar, Rishabh Jain, Marco Battiato
Study of far-from-equilibrium thermalization dynamics in quantum materials, including the dynamics of different types of quasiparticles, is becoming increasingly crucial. However,…
Contrast and Classify: Training Robust VQA Models
Yash Kant, Abhinav Moudgil, Dhruv Batra +2
Recent Visual Question Answering (VQA) models have shown impressive performance on the VQA benchmark but remain sensitive to small linguistic variations in input questions. Existin…
Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Views
Vincent Cartillier, Zhile Ren, Neha Jain +3
We study the task of semantic mapping - specifically, an embodied agent (a robot or an egocentric AI assistant) is given a tour of a new environment and asked to build an allocentr…
Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents
Samyak Datta, Oleksandr Maksymets, Judy Hoffman +3
Recent work has presented embodied agents that can navigate to point-goal targets in novel indoor environments with near-perfect accuracy. However, these agents are equipped with i…
Dialog without Dialog Data: Learning Visual Dialog Agents from VQA Data
Michael Cogswell, Jiasen Lu, Rishabh Jain +3
Can we develop visually grounded dialog agents that can efficiently adapt to new tasks without forgetting how to talk to people? Such agents could leverage a larger variety of exis…
Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation
Medhini Narasimhan, Erik Wijmans, Xinlei Chen +4
We introduce a learning-based approach for room navigation using semantic maps. Our proposed architecture learns to predict top-down belief maps of regions that lie beyond the agen…
Spatially Aware Multimodal Transformers for TextVQA
Yash Kant, Dhruv Batra, Peter Anderson +4
Textual cues are essential for everyday tasks like buying groceries and using public transport. To develop this assistive technology, we study the TextVQA task, i.e., reasoning abo…
Auxiliary Tasks Speed Up Learning PointGoal Navigation
Joel Ye, Dhruv Batra, Erik Wijmans +1
PointGoal Navigation is an embodied task that requires agents to navigate to a specified point in an unseen environment. Wijmans et al. showed that this task is solvable but their…
ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects
Dhruv Batra, Aaron Gokaslan, Aniruddha Kembhavi +5
We revisit the problem of Object-Goal Navigation (ObjectNav). In its simplest form, ObjectNav is defined as the task of navigating to an object, specified by its label, in an unexp…
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web
Arjun Majumdar, Ayush Shrivastava, Stefan Lee +3
Following a navigation instruction such as 'Walk down the stairs and stop at the brown sofa' requires embodied AI agents to ground scene elements referenced via language (e.g. 'sta…
Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments
Jacob Krantz, Erik Wijmans, Arjun Majumdar +2
We develop a language-guided navigation task set in a continuous 3D environment where agents must execute low-level actions to follow natural language navigation directions. By bei…
Analyzing Visual Representations in Embodied Navigation Tasks
Erik Wijmans, Julian Straub, Dhruv Batra +3
Recent advances in deep reinforcement learning require a large amount of training data and generally result in representations that are often over specialized to the target task. I…
Recurrent Dirichlet Belief Networks for Interpretable Dynamic Relational Data Modelling
Yaqiong Li, Xuhui Fan, Ling Chen +3
The Dirichlet Belief Network~(DirBN) has been recently proposed as a promising approach in learning interpretable deep latent representations for objects. In this work, we leverage…