24 citations · 127 across the 27 of their papers we have counts for
25 papers · 1 filter
Compositional Planning with Jumpy World Models
Jesse Farebrother, Matteo Pirotta, Andrea Tirinzoni +3
The ability to plan with temporal abstractions is central to intelligent decision-making. Rather than reasoning over primitive actions, we study agents that compose pre-trained pol…
TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learning
Marco Bagatella, Matteo Pirotta, Ahmed Touati +2
Latent prediction--where agents learn by predicting their own latents--has emerged as a powerful paradigm for training general representations in machine learning. In reinforcement…
Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models
Andrea Tirinzoni, Ahmed Touati, Jesse Farebrother +5
Unsupervised reinforcement learning (RL) aims at pre-training agents that can solve a wide range of downstream tasks in complex environments. Despite recent advancements, existing…
Fast Adaptation with Behavioral Foundation Models
Harshit Sikchi, Andrea Tirinzoni, Ahmed Touati +6
Unsupervised zero-shot reinforcement learning (RL) has emerged as a powerful paradigm for pretraining behavioral foundation models (BFMs), enabling agents to solve a wide range of…
Temporal Difference Flows
Jesse Farebrother, Matteo Pirotta, Andrea Tirinzoni +3
Predictive models of the future are fundamental for an agent's ability to reason and plan. A common strategy learns a world model and unrolls it step-by-step at inference, where sm…
Finer Behavioral Foundation Models via Auto-Regressive Features and Advantage Weighting
Edoardo Cetin, Ahmed Touati, Yann Ollivier
The forward-backward representation (FB) is a recently proposed framework (Touati et al., 2023; Touati & Ollivier, 2021) to train behavior foundation models (BFMs) that aim at prov…
Simple Ingredients for Offline Reinforcement Learning
Edoardo Cetin, Andrea Tirinzoni, Matteo Pirotta +3
Offline reinforcement learning algorithms have proven effective on datasets highly connected to the target downstream task. Yet, leveraging a novel testbed (MOOD) in which trajecto…
SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learning
Harshit Sikchi, Rohan Chitnis, Ahmed Touati +3
Offline Goal-Conditioned Reinforcement Learning (GCRL) is tasked with learning to achieve multiple goals in an environment purely from offline datasets using sparse reward function…
A State Representation for Diminishing Rewards
Ted Moskovitz, Samo Hromadka, Ahmed Touati +2
A common setting in multitask reinforcement learning (RL) demands that an agent rapidly adapt to various stationary reward functions randomly sampled from a fixed distribution. In…
Scalable Representation Learning in Linear Contextual Bandits with Constant Regret Guarantees
Andrea Tirinzoni, Matteo Papini, Ahmed Touati +2
We study the problem of representation learning in stochastic contextual linear bandits. While the primary concern in this domain is usually to find realizable representations (i.e…
Does Zero-Shot Reinforcement Learning Exist?
Ahmed Touati, Jérémy Rapin, Yann Ollivier
A zero-shot RL agent is an agent that can solve any RL task in a given environment, instantly with no additional planning or learning, after an initial reward-free learning phase.…
Learning One Representation to Optimize All Rewards
Ahmed Touati, Yann Ollivier
We introduce the forward-backward (FB) representation of the dynamics of a reward-free Markov decision process. It provides explicit near-optimal policies for any reward specified…
Efficient Learning in Non-Stationary Linear Markov Decision Processes
Ahmed Touati, Pascal Vincent
We study episodic reinforcement learning in non-stationary linear (a.k.a. low-rank) Markov Decision Processes (MDPs), i.e, both the reward and transition kernel are linear with res…
Maximum Reward Formulation In Reinforcement Learning
Sai Krishna Gottipati, Yashaswi Pathak, Rohan Nuttall +6
Reinforcement learning (RL) algorithms typically deal with maximizing the expected cumulative return (discounted or undiscounted, finite or infinite horizon). However, several cruc…
Sharp Analysis of Smoothed Bellman Error Embedding
Ahmed Touati, Pascal Vincent
The \textit{Smoothed Bellman Error Embedding} algorithm~\citep{dai2018sbeed}, known as SBEED, was proposed as a provably convergent reinforcement learning algorithm with general no…
TDprop: Does Jacobi Preconditioning Help Temporal Difference Learning?
Joshua Romoff, Peter Henderson, David Kanaa +4
We investigate whether Jacobi preconditioning, accounting for the bootstrap term in temporal difference (TD) learning, can help boost performance of adaptive optimizers. Our method…
Zooming for Efficient Model-Free Reinforcement Learning in Metric Spaces
Ahmed Touati, Adrien Ali Taiga, Marc G. Bellemare
Despite the wealth of research into provably efficient reinforcement learning algorithms, most works focus on tabular representation and thus struggle to handle exponentially or in…
Stable Policy Optimization via Off-Policy Divergence Regularization
Ahmed Touati, Amy Zhang, Joelle Pineau +1
Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) are among the most successful policy gradient approaches in deep reinforcement learning (RL). While t…
Stochastic Neural Network with Kronecker Flow
Chin-Wei Huang, Ahmed Touati, Pascal Vincent +3
Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to scale to the high-dimensional setting o…
SVRG for Policy Evaluation with Fewer Gradient Evaluations
Zilun Peng, Ahmed Touati, Pascal Vincent +1
Stochastic variance-reduced gradient (SVRG) is an optimization method originally designed for tackling machine learning problems with a finite sum structure. SVRG was later shown t…
Separating value functions across time-scales
Joshua Romoff, Peter Henderson, Ahmed Touati +3
In many finite horizon episodic reinforcement learning (RL) settings, it is desirable to optimize for the undiscounted return - in settings like Atari, for instance, the goal is to…
Randomized Value Functions via Multiplicative Normalizing Flows
Ahmed Touati, Harsh Satija, Joshua Romoff +2
Randomized value functions offer a promising approach towards the challenge of efficient exploration in complex environments with high dimensional state and action spaces. Unlike t…
Learnable Explicit Density for Continuous Latent Space and Variational Inference
Chin-Wei Huang, Ahmed Touati, Laurent Dinh +4
In this paper, we study two aspects of the variational autoencoder (VAE): the prior distribution over the latent variables and its corresponding posterior. First, we decompose the…
Parametric Adversarial Divergences are Good Losses for Generative Modeling
Gabriel Huang, Hugo Berard, Ahmed Touati +3
Parametric adversarial divergences, which are a generalization of the losses used to train generative adversarial networks (GANs), have often been described as being approximations…