32 citations · 108 across the 28 of their papers we have counts for
30 papers
Imagining Goals with Geometric Horizon Models for Policy Generalization
Daniel Lawson, Dhruv Sreenivas, Adriana Hugessen +2
Training from offline data has allowed for substantial progress in domains such as robotics, leading to general-purpose policies that can be easily applied zero-shot or efficiently…
Difference-Aware Retrieval Policies for Imitation Learning
Quinn Pfeifer, Ethan Pronovost, Paarth Shah +3
Parametric imitation learning via behavior cloning can suffer from poor generalization to out-of-distribution states due to compounding errors during deployment. We show that reusi…
Preventing Learning Stagnation in PPO by Scaling to 1 Million Parallel Environments
Michael Beukman, Khimya Khetarpal, Zeyu Zheng +4
An agent's performance stagnating at a suboptimal level is a common problem in deep on-policy RL. Focusing on PPO, we show that plateaus in certain regimes arise not because of kno…
Affordances Enable Partial World Modeling with LLMs
Khimya Khetarpal, Gheorghe Comanici, Jonathan Richens +5
Full models of the world require complex knowledge of immense detail. While pre-trained large models have been hypothesized to contain similar knowledge due to extensive pre-traini…
The Geometry of Learning to Avoid Interventions
Ethan Pronovost, Khimya Khetarpal, Siddhartha Srinivasa
Human interventions are a common source of supervision in autonomous systems during deployment. Many existing approaches are based on avoiding interventions, yet the consequences o…
Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloning
Daniel Lawson, Adriana Hugessen, Charlotte Cloutier +2
While goal-conditioned behavior cloning (GCBC) methods can perform well on in-distribution training tasks, they do not necessarily generalize zero-shot to tasks that require condit…
Plasticity as the Mirror of Empowerment
David Abel, Michael Bowling, André Barreto +13
Agents are minimally entities that are influenced by their past observations and act to influence future observations. This latter capacity is captured by empowerment, which has se…
Cracking the Code of Action: a Generative Approach to Affordances for Reinforcement Learning
Lynn Cherif, Flemming Kondrup, David Venuto +3
Agents that can autonomously navigate the web through a graphical user interface (GUI) using a unified action space (e.g., mouse and keyboard actions) can require very large amount…
Representation Learning via Non-Contrastive Mutual Information
Zhaohan Daniel Guo, Bernardo Avila Pires, Khimya Khetarpal +2
Labeling data is often very time consuming and expensive, leaving us with a majority of unlabeled data. Self-supervised representation learning methods such as SimCLR (Chen et al.,…
Long Range Navigator (LRN): Extending robot planning horizons beyond metric maps
Matt Schmittle, Rohan Baijal, Nathan Hatch +6
A robot navigating an outdoor environment with no prior knowledge of the space must rely on its local sensing to perceive its surroundings and plan. This can come in the form of a…
Agency Is Frame-Dependent
David Abel, André Barreto, Michael Bowling +13
Agency is a system's capacity to steer outcomes toward a goal, and is a central topic of study across biology, philosophy, cognitive science, and artificial intelligence. Determini…
Optimizing Return Distributions with Distributional Dynamic Programming
Bernardo Ávila Pires, Mark Rowland, Diana Borsa +6
We introduce distributional dynamic programming (DP) methods for optimizing statistical functionals of the return distribution, with standard reinforcement learning as a special ca…
Normalization and effective learning rates in reinforcement learning
Clare Lyle, Zeyu Zheng, Khimya Khetarpal +4
Normalization layers have recently experienced a renaissance in the deep reinforcement learning and continual learning literature, with several works highlighting diverse benefits…
A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning
Khimya Khetarpal, Zhaohan Daniel Guo, Bernardo Avila Pires +7
Learning a good representation is a crucial challenge for Reinforcement Learning (RL) agents. Self-predictive learning provides means to jointly learn a latent representation and d…
Disentangling the Causes of Plasticity Loss in Neural Networks
Clare Lyle, Zeyu Zheng, Khimya Khetarpal +4
Underpinning the past decades of work on the design, initialization, and optimization of neural networks is a seemingly innocuous assumption: that the network is trained on a \text…
Toward Human-AI Alignment in Large-Scale Multi-Player Games
Sugandha Sharma, Guy Davidson, Khimya Khetarpal +4
Achieving human-AI alignment in complex multi-agent games is crucial for creating trustworthy AI agents that enhance gameplay. We propose a method to evaluate this alignment using…
Forecaster: Towards Temporally Abstract Tree-Search Planning from Pixels
Thomas Jiralerspong, Flemming Kondrup, Doina Precup +1
The ability to plan at many different levels of abstraction enables agents to envision the long-term repercussions of their decisions and thus enables sample-efficient learning. Th…
Discovering Object-Centric Generalized Value Functions From Pixels
Somjit Nath, Gopeshh Raaj Subbaraj, Khimya Khetarpal +1
Deep Reinforcement Learning has shown significant progress in extracting useful representations from high-dimensional inputs albeit using hand-crafted auxiliary tasks and pseudo re…
POMRL: No-Regret Learning-to-Plan with Increasing Horizons
Khimya Khetarpal, Claire Vernade, Brendan O'Donoghue +2
We study the problem of planning under model uncertainty in an online meta-reinforcement learning (RL) setting where an agent is presented with a sequence of related tasks with lim…
The Paradox of Choice: Using Attention in Hierarchical Reinforcement Learning
Andrei Nica, Khimya Khetarpal, Doina Precup
Decision-making AI agents are often faced with two important challenges: the depth of the planning horizon, and the branching factor due to having many choices. Hierarchical reinfo…
Temporally Abstract Partial Models
Khimya Khetarpal, Zafarali Ahmed, Gheorghe Comanici +1
Humans and animals have the ability to reason and make predictions about different courses of action at many time scales. In reinforcement learning, option models (Sutton, Precup \…
Sequoia: A Software Framework to Unify Continual Learning Research
Fabrice Normandin, Florian Golemo, Oleksiy Ostapenko +10
The field of Continual Learning (CL) seeks to develop algorithms that accumulate knowledge and skills over time through interaction with non-stationary environments. In practice, a…
Variance Penalized On-Policy and Off-Policy Actor-Critic
Arushi Jain, Gandharv Patil, Ayush Jain +2
Reinforcement learning algorithms are typically geared towards optimizing the expected return of an agent. However, in many practical applications, low variance in the return is de…
Towards Continual Reinforcement Learning: A Review and Perspectives
Khimya Khetarpal, Matthew Riemer, Irina Rish +1
In this article, we aim to provide a literature review of different formulations and approaches to continual reinforcement learning (RL), also known as lifelong or non-stationary R…