368 citations · 860 across the 19 of their papers we have counts for
20 papers
Hierarchical reinforcement learning with natural language subgoals
Arun Ahuja, Kavya Kopparapu, Rob Fergus +1
Hierarchical reinforcement learning has been a compelling approach for achieving goal directed behavior over long sequences of actions. However, it has been challenging to implemen…
Collaborating with language models for embodied reasoning
Ishita Dasgupta, Christine Kaeser-Chen, Kenneth Marino +4
Reasoning in a complex and ambiguous environment is a key goal for Reinforcement Learning (RL) agents. While some sophisticated RL agents can successfully solve difficult tasks, th…
Distilling Internet-Scale Vision-Language Models into Embodied Agents
Theodore Sumers, Kenneth Marino, Arun Ahuja +2
Instruction-following agents must ground language into their observation and action spaces. Learning to ground language is challenging, typically requiring domain-specific engineer…
Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedback
Josh Abramson, Arun Ahuja, Federico Carnevale +16
An important goal in artificial intelligence is to create agents that can both interact naturally with humans and learn from their feedback. Here we demonstrate how to use reinforc…
Learning to Navigate Wikipedia by Taking Random Walks
Manzil Zaheer, Kenneth Marino, Will Grathwohl +7
A fundamental ability of an intelligent web-based agent is seeking out and acquiring new information. Internet search engines reliably find the correct vicinity but the top results…
Evaluating Multimodal Interactive Agents
Josh Abramson, Arun Ahuja, Federico Carnevale +12
Creating agents that can interact naturally with humans is a common goal in artificial intelligence (AI) research. However, evaluating these interactions is challenging: collecting…
Dynamical crossover behavior in the relaxation of quenched quantum many-body systems
Aamir Ahmad Makki, Souvik Bandyopadhyay, Somnath Maity +1
A crossover between different power-law relaxation behaviors of many-body periodically driven integrable systems has come to light in recent years. We demonstrate using integrable…
Imitation by Predicting Observations
Andrew Jaegle, Yury Sulsky, Arun Ahuja +3
Imitation learning enables agents to reuse and adapt the hard-won expertise of others, offering a solution to several key challenges in learning behavior. Although it is easy to ob…
Imitating Interactive Intelligence
Josh Abramson, Arun Ahuja, Iain Barr +26
A common vision from science fiction is that robots will one day inhabit our physical spaces, sense the world as we do, assist our physical labours, and communicate with us through…
Behavior Priors for Efficient Reinforcement Learning
Dhruva Tirumala, Alexandre Galashov, Hyeonwoo Noh +8
As we deploy reinforcement learning agents to solve increasingly challenging problems, methods that allow us to inject prior knowledge about the structure of the world and effectiv…
Probing Emergent Semantics in Predictive Agents via Question Answering
Abhishek Das, Federico Carnevale, Hamza Merzic +8
Recent work has shown how predictive modeling can endow agents with rich knowledge of their surroundings, improving their ability to act in complex environments. We propose questio…
Catch & Carry: Reusable Neural Controllers for Vision-Guided Whole-Body Tasks
Josh Merel, Saran Tunyasuvunakool, Arun Ahuja +6
We address the longstanding challenge of producing flexible, realistic humanoid character controllers that can perform diverse whole-body tasks involving object interactions. This…
V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Control
H. Francis Song, Abbas Abdolmaleki, Jost Tobias Springenberg +11
Some of the most successful applications of deep reinforcement learning to challenging domains in discrete and continuous control have used policy gradient methods in the on-policy…
Exploiting Hierarchy for Learning and Transfer in KL-regularized RL
Dhruva Tirumala, Hyeonwoo Noh, Alexandre Galashov +6
As reinforcement learning agents are tasked with solving more challenging and diverse tasks, the ability to incorporate prior knowledge into the learning system and to exploit reus…
Neural probabilistic motor primitives for humanoid control
Josh Merel, Leonard Hasenclever, Alexandre Galashov +5
We focus on the problem of learning a single motor module that can flexibly express a range of behaviors for the control of high-dimensional physically simulated humanoids. To do t…
Experience Replay for Continual Learning
David Rolnick, Arun Ahuja, Jonathan Schwarz +2
Continual learning is the problem of learning new tasks or knowledge while protecting old knowledge and ideally generalizing from old experience to learn new tasks faster. Neural n…
Hierarchical visuomotor control of humanoids
Josh Merel, Arun Ahuja, Vu Pham +5
We aim to build complex humanoid agents that integrate perception, motor control, and memory. In this work, we partly factor this problem into low-level motor control from proprioc…
Optimizing Agent Behavior over Long Time Scales by Transporting Value
Chia-Chun Hung, Timothy Lillicrap, Josh Abramson +5
Humans spend a remarkable fraction of waking life engaged in acts of "mental time travel". We dwell on our actions in the past and experience satisfaction or regret. More than mere…
Probing Physics Knowledge Using Tools from Developmental Psychology
Luis Piloto, Ari Weinstein, Dhruva TB +6
In order to build agents with a rich understanding of their environment, one key objective is to endow them with a grasp of intuitive physics; an ability to reason about three-dime…
Unsupervised Predictive Memory in a Goal-Directed Agent
Greg Wayne, Chia-Chun Hung, David Amos +21
Animals execute goal-directed behaviours despite the limited range and scope of their sensors. To cope, they explore environments and store memories maintaining estimates of import…