26 citations · 60 across the 25 of their papers we have counts for
27 papers
Compositional Planning with Jumpy World Models
Jesse Farebrother, Matteo Pirotta, Andrea Tirinzoni +3
The ability to plan with temporal abstractions is central to intelligent decision-making. Rather than reasoning over primitive actions, we study agents that compose pre-trained pol…
BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning
Yitang Li, Zhengyi Luo, Tonghe Zhang +10
Building Behavioral Foundation Models (BFMs) for humanoid robots has the potential to unify diverse control tasks under a single, promptable generalist policy. However, existing ap…
TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learning
Marco Bagatella, Matteo Pirotta, Ahmed Touati +2
Latent prediction--where agents learn by predicting their own latents--has emerged as a powerful paradigm for training general representations in machine learning. In reinforcement…
Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models
Andrea Tirinzoni, Ahmed Touati, Jesse Farebrother +5
Unsupervised reinforcement learning (RL) aims at pre-training agents that can solve a wide range of downstream tasks in complex environments. Despite recent advancements, existing…
Fast Adaptation with Behavioral Foundation Models
Harshit Sikchi, Andrea Tirinzoni, Ahmed Touati +6
Unsupervised zero-shot reinforcement learning (RL) has emerged as a powerful paradigm for pretraining behavioral foundation models (BFMs), enabling agents to solve a wide range of…
Temporal Difference Flows
Jesse Farebrother, Matteo Pirotta, Andrea Tirinzoni +3
Predictive models of the future are fundamental for an agent's ability to reason and plan. A common strategy learns a world model and unrolls it step-by-step at inference, where sm…
Simple Ingredients for Offline Reinforcement Learning
Edoardo Cetin, Andrea Tirinzoni, Matteo Pirotta +3
Offline reinforcement learning algorithms have proven effective on datasets highly connected to the target downstream task. Yet, leveraging a novel testbed (MOOD) in which trajecto…
Towards Instance-Optimality in Online PAC Reinforcement Learning
Aymen Al-Marjani, Andrea Tirinzoni, Emilie Kaufmann
Several recent works have proposed instance-dependent upper bounds on the number of episodes needed to identify, with probability , an -optimal policy in finite-h…
Active Coverage for PAC Reinforcement Learning
Aymen Al-Marjani, Andrea Tirinzoni, Emilie Kaufmann
Collecting and leveraging data with good coverage properties plays a crucial role in different aspects of reinforcement learning (RL), including reward-free exploration and offline…
Layered State Discovery for Incremental Autonomous Exploration
Liyu Chen, Andrea Tirinzoni, Alessandro Lazaric +1
We study the autonomous exploration (AX) problem proposed by Lim & Auer (2012). In this setting, the objective is to discover a set of -optimal policies reaching a set $\mathcal…
On the Complexity of Representation Learning in Contextual Linear Bandits
Andrea Tirinzoni, Matteo Pirotta, Alessandro Lazaric
In contextual linear bandits, the reward function is assumed to be a linear combination of an unknown reward vector and a given embedding of context-arm pairs. In practice, the emb…
Scalable Representation Learning in Linear Contextual Bandits with Constant Regret Guarantees
Andrea Tirinzoni, Matteo Papini, Ahmed Touati +2
We study the problem of representation learning in stochastic contextual linear bandits. While the primary concern in this domain is usually to find realizable representations (i.e…
Reaching Goals is Hard: Settling the Sample Complexity of the Stochastic Shortest Path
Liyu Chen, Andrea Tirinzoni, Matteo Pirotta +1
We study the sample complexity of learning an -optimal policy in the Stochastic Shortest Path (SSP) problem. We first derive sample complexity bounds when the learner has access…
Optimistic PAC Reinforcement Learning: the Instance-Dependent View
Andrea Tirinzoni, Aymen Al-Marjani, Emilie Kaufmann
Optimistic algorithms have been extensively studied for regret minimization in episodic tabular MDPs, both from a minimax and an instance-dependent view. However, for the PAC RL pr…
On Elimination Strategies for Bandit Fixed-Confidence Identification
Andrea Tirinzoni, Rémy Degenne
Elimination algorithms for bandit identification, which prune the plausible correct answers sequentially until only one remains, are computationally convenient since they reduce th…
Near Instance-Optimal PAC Reinforcement Learning for Deterministic MDPs
Andrea Tirinzoni, Aymen Al-Marjani, Emilie Kaufmann
In probably approximately correct (PAC) reinforcement learning (RL), an agent is required to identify an -optimal policy with probability . While minimax optimal algorithms…
Dealing With Misspecification In Fixed-Confidence Linear Top-m Identification
Clémence Réda, Andrea Tirinzoni, Rémy Degenne
We study the problem of the identification of m arms with largest means under a fixed error rate (fixed-confidence Top-m identification), for misspecified linear bandit models.…
Reinforcement Learning in Linear MDPs: Constant Regret and Representation Selection
Matteo Papini, Andrea Tirinzoni, Aldo Pacchiano +3
We study the role of the representation of state-action value functions in regret minimization in finite-horizon Markov Decision Processes (MDPs) with linear structure. We first de…
A Fully Problem-Dependent Regret Lower Bound for Finite-Horizon MDPs
Andrea Tirinzoni, Matteo Pirotta, Alessandro Lazaric
We derive a novel asymptotic problem-dependent lower-bound for regret minimization in finite-horizon tabular Markov Decision Processes (MDPs). While, similar to prior work (e.g., f…
Meta-Reinforcement Learning by Tracking Task Non-stationarity
Riccardo Poiani, Andrea Tirinzoni, Marcello Restelli
Many real-world domains are subject to a structured non-stationarity which affects the agent's goals and the environmental dynamics. Meta-reinforcement learning (RL) has been shown…
Leveraging Good Representations in Linear Contextual Bandits
Matteo Papini, Andrea Tirinzoni, Marcello Restelli +2
The linear contextual bandit literature is mostly focused on the design of efficient learning algorithms for a given representation. However, a contextual bandit problem may admit…
An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear Bandits
Andrea Tirinzoni, Matteo Pirotta, Marcello Restelli +1
In the contextual linear bandit setting, algorithms built on the optimism principle fail to exploit the structure of the problem and have been shown to be asymptotically suboptimal…
Sequential Transfer in Reinforcement Learning with a Generative Model
Andrea Tirinzoni, Riccardo Poiani, Marcello Restelli
We are interested in how to design reinforcement learning agents that provably reduce the sample complexity for learning new tasks by transferring knowledge from previously-solved…
A Novel Confidence-Based Algorithm for Structured Bandits
Andrea Tirinzoni, Alessandro Lazaric, Marcello Restelli
We study finite-armed stochastic bandits where the rewards of each arm might be correlated to those of other arms. We introduce a novel phased algorithm that exploits the given str…