32 citations · 108 across the 28 of their papers we have counts for
19 papers · 1 filter
Imagining Goals with Geometric Horizon Models for Policy Generalization
Daniel Lawson, Dhruv Sreenivas, Adriana Hugessen +2
Training from offline data has allowed for substantial progress in domains such as robotics, leading to general-purpose policies that can be easily applied zero-shot or efficiently…
Preventing Learning Stagnation in PPO by Scaling to 1 Million Parallel Environments
Michael Beukman, Khimya Khetarpal, Zeyu Zheng +4
An agent's performance stagnating at a suboptimal level is a common problem in deep on-policy RL. Focusing on PPO, we show that plateaus in certain regimes arise not because of kno…
Affordances Enable Partial World Modeling with LLMs
Khimya Khetarpal, Gheorghe Comanici, Jonathan Richens +5
Full models of the world require complex knowledge of immense detail. While pre-trained large models have been hypothesized to contain similar knowledge due to extensive pre-traini…
The Geometry of Learning to Avoid Interventions
Ethan Pronovost, Khimya Khetarpal, Siddhartha Srinivasa
Human interventions are a common source of supervision in autonomous systems during deployment. Many existing approaches are based on avoiding interventions, yet the consequences o…
Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloning
Daniel Lawson, Adriana Hugessen, Charlotte Cloutier +2
While goal-conditioned behavior cloning (GCBC) methods can perform well on in-distribution training tasks, they do not necessarily generalize zero-shot to tasks that require condit…
Representation Learning via Non-Contrastive Mutual Information
Zhaohan Daniel Guo, Bernardo Avila Pires, Khimya Khetarpal +2
Labeling data is often very time consuming and expensive, leaving us with a majority of unlabeled data. Self-supervised representation learning methods such as SimCLR (Chen et al.,…
Optimizing Return Distributions with Distributional Dynamic Programming
Bernardo Ávila Pires, Mark Rowland, Diana Borsa +6
We introduce distributional dynamic programming (DP) methods for optimizing statistical functionals of the return distribution, with standard reinforcement learning as a special ca…
Normalization and effective learning rates in reinforcement learning
Clare Lyle, Zeyu Zheng, Khimya Khetarpal +4
Normalization layers have recently experienced a renaissance in the deep reinforcement learning and continual learning literature, with several works highlighting diverse benefits…
A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning
Khimya Khetarpal, Zhaohan Daniel Guo, Bernardo Avila Pires +7
Learning a good representation is a crucial challenge for Reinforcement Learning (RL) agents. Self-predictive learning provides means to jointly learn a latent representation and d…
Disentangling the Causes of Plasticity Loss in Neural Networks
Clare Lyle, Zeyu Zheng, Khimya Khetarpal +4
Underpinning the past decades of work on the design, initialization, and optimization of neural networks is a seemingly innocuous assumption: that the network is trained on a \text…
Discovering Object-Centric Generalized Value Functions From Pixels
Somjit Nath, Gopeshh Raaj Subbaraj, Khimya Khetarpal +1
Deep Reinforcement Learning has shown significant progress in extracting useful representations from high-dimensional inputs albeit using hand-crafted auxiliary tasks and pseudo re…
The Paradox of Choice: Using Attention in Hierarchical Reinforcement Learning
Andrei Nica, Khimya Khetarpal, Doina Precup
Decision-making AI agents are often faced with two important challenges: the depth of the planning horizon, and the branching factor due to having many choices. Hierarchical reinfo…
Temporally Abstract Partial Models
Khimya Khetarpal, Zafarali Ahmed, Gheorghe Comanici +1
Humans and animals have the ability to reason and make predictions about different courses of action at many time scales. In reinforcement learning, option models (Sutton, Precup \…
Sequoia: A Software Framework to Unify Continual Learning Research
Fabrice Normandin, Florian Golemo, Oleksiy Ostapenko +10
The field of Continual Learning (CL) seeks to develop algorithms that accumulate knowledge and skills over time through interaction with non-stationary environments. In practice, a…
Variance Penalized On-Policy and Off-Policy Actor-Critic
Arushi Jain, Gandharv Patil, Ayush Jain +2
Reinforcement learning algorithms are typically geared towards optimizing the expected return of an agent. However, in many practical applications, low variance in the return is de…
Towards Continual Reinforcement Learning: A Review and Perspectives
Khimya Khetarpal, Matthew Riemer, Irina Rish +1
In this article, we aim to provide a literature review of different formulations and approaches to continual reinforcement learning (RL), also known as lifelong or non-stationary R…
Learning Robust State Abstractions for Hidden-Parameter Block MDPs
Amy Zhang, Shagun Sodhani, Khimya Khetarpal +1
Many control tasks exhibit similar dynamics that can be modeled as having common latent structure. Hidden-Parameter Markov Decision Processes (HiP-MDPs) explicitly model this struc…
What can I do here? A Theory of Affordances in Reinforcement Learning
Khimya Khetarpal, Zafarali Ahmed, Gheorghe Comanici +2
Reinforcement learning algorithms usually assume that all actions are always available to an agent. However, both people and animals understand the general link between the feature…
Options of Interest: Temporal Abstraction with Interest Functions
Khimya Khetarpal, Martin Klissarov, Maxime Chevalier-Boisvert +2
Temporal abstraction refers to the ability of an agent to use behaviours of controllers which act for a limited, variable amount of time. The options framework describes such behav…