Publications (13)
A Generalist Agent
Scott Reed, Konrad Zolna, Emilio Parisotto +17
Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we…
Learning Deployable Navigation Policies at Kilometer Scale from a Single Traversal
Jake Bruce, Niko Sünderhauf, Piotr Mirowski +2
Model-free reinforcement learning has recently been shown to be effective at learning navigation policies from complex image input. However, these algorithms tend to require large…
Look No Further: Adapting the Localization Sensory Window to the Temporal Characteristics of the Environment
Jake Bruce, Adam Jacobson, Michael Milford
Many localization algorithms use a spatiotemporal window of sensory information in order to recognize spatial locations, and the length of this window is often a sensitive paramete…
A Generalist Dynamics Model for Control
Ingmar Schubert, Jingwei Zhang, Jake Bruce +7
We investigate the use of transformer sequence models as dynamics models (TDMs) for control. We find that TDMs exhibit strong generalization capabilities to unseen environments, bo…
One-Shot Reinforcement Learning for Robot Navigation with Interactive Replay
Jake Bruce, Niko Suenderhauf, Piotr Mirowski +2
Recently, model-free reinforcement learning algorithms have been shown to solve challenging problems by learning from extensive interaction with the environment. A significant issu…
Video as the New Language for Real-World Decision Making
Sherry Yang, Jacob Walker, Jack Parker-Holder +5
Both text and video data are abundant on the internet and support large-scale self-supervised learning through next token or frame prediction. However, they have not been equally l…
Evaluating task-agnostic exploration for fixed-batch learning of arbitrary future tasks
Vibhavari Dasagi, Robert Lee, Jake Bruce +1
Deep reinforcement learning has been shown to solve challenging tasks where large amounts of training experience is available, usually obtained online while learning the task. Robo…
Accelerating exploration and representation learning with offline pre-training
Bogdan Mazoure, Jake Bruce, Doina Precup +2
Sequential decision-making agents struggle with long horizon tasks, since solving them requires multi-step reasoning. Most reinforcement learning (RL) algorithms address this chall…
Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments
Peter Anderson, Qi Wu, Damien Teney +6
A robot that can carry out a natural-language instruction has been a dream since before the Jetsons cartoon series imagined a life of leisure mediated by a fleet of attentive robot…
Ctrl-Z: Recovering from Instability in Reinforcement Learning
Vibhavari Dasagi, Jake Bruce, Thierry Peynot +1
When learning behavior, training data is often generated by the learner itself; this can result in unstable training dynamics, and this problem has particularly important applicati…
Genie: Generative Interactive Environments
Jake Bruce, Michael Dennis, Ashley Edwards +22
We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless…
Sim-to-Real Transfer of Robot Learning with Variable Length Inputs
Vibhavari Dasagi, Robert Lee, Serena Mou +3
Current end-to-end deep Reinforcement Learning (RL) approaches require jointly learning perception, decision-making and low-level control from very sparse reward signals and high-d…
Imitation by Predicting Observations
Andrew Jaegle, Yury Sulsky, Arun Ahuja +3
Imitation learning enables agents to reuse and adapt the hard-won expertise of others, offering a solution to several key challenges in learning behavior. Although it is easy to ob…