3 citations · 11 across the 26 of their papers we have counts for
28 papers
Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Models
Shelly Francis-Meretzki, Mirco Mutti, Yaniv Romano +1
Recent advances in vision-language-action (VLA) models for robotics have highlighted the importance of reliable uncertainty quantification in sequential tasks. However, assessing a…
Unsupervised Behavioral Compression: Learning Low-Dimensional Policy Manifolds through State-Occupancy Matching
Andrea Fraschini, Davide Tenedini, Riccardo Zamboni +2
Deep Reinforcement Learning (DRL) is widely recognized as sample-inefficient, a limitation attributable in part to the high dimensionality and substantial functional redundancy inh…
K-Myriad: Jump-starting reinforcement learning with unsupervised parallel agents
Vincenzo De Paola, Mirco Mutti, Riccardo Zamboni +1
Parallelization in Reinforcement Learning is typically employed to speed up the training of a single policy, where multiple workers collect experience from an identical sampling di…
Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogames
Ev Zisselman, Mirco Mutti, Shelly Francis-Meretzki +2
Behavioral cloning is a simple yet effective technique for learning sequential decision-making from demonstrations. Recently, it has gained prominence as the core of foundation mod…
From Parameters to Behaviors: Unsupervised Compression of the Policy Space
Davide Tenedini, Riccardo Zamboni, Mirco Mutti +1
Despite its recent successes, Deep Reinforcement Learning (DRL) is notoriously sample-inefficient. We argue that this inefficiency stems from the standard practice of optimizing po…
Enhancing Diversity in Parallel Agents: A Maximum State Entropy Exploration Story
Vincenzo De Paola, Riccardo Zamboni, Mirco Mutti +1
Parallel data collection has redefined Reinforcement Learning (RL), unlocking unprecedented efficiency and powering breakthroughs in large-scale real-world applications. In this pa…
State Entropy Regularization for Robust Reinforcement Learning
Yonatan Ashlag, Uri Koren, Mirco Mutti +3
State entropy regularization has empirically shown better exploration and sample complexity in reinforcement learning (RL). However, its theoretical guarantees have not been studie…
A Classification View on Meta Learning Bandits
Mirco Mutti, Jeongyeol Kwon, Shie Mannor +1
Contextual multi-armed bandits are a popular choice to model sequential decision-making. E.g., in a healthcare application we may perform various tests to asses a patient condition…
Towards Principled Unsupervised Multi-Agent Reinforcement Learning
Riccardo Zamboni, Mirco Mutti, Marcello Restelli
In reinforcement learning, we typically refer to unsupervised pre-training when we aim to pre-train a policy without a priori access to the task specification, i.e. rewards, to be…
Reward Compatibility: A Framework for Inverse RL
Filippo Lazzati, Mirco Mutti, Alberto Metelli
We provide an original theoretical study of Inverse Reinforcement Learning (IRL) through the lens of reward compatibility, a novel framework to quantify the compatibility of a rewa…
Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstraction
Riccardo De Santi, Federico Arangath Joseph, Noah Liniger +2
How can a scientist use a Reinforcement Learning (RL) algorithm to design experiments over a dynamical system's state space? In the case of finite and Markovian systems, an area ca…
The Limits of Pure Exploration in POMDPs: When the Observation Entropy is Enough
Riccardo Zamboni, Duilio Cirino, Marcello Restelli +1
The problem of pure exploration in Markov decision processes has been cast as maximizing the entropy over the state distribution induced by the agent's policy, an objective that ha…
How to Explore with Belief: State Entropy Maximization in POMDPs
Riccardo Zamboni, Duilio Cirino, Marcello Restelli +1
Recent works have studied *state entropy maximization* in reinforcement learning, in which the agent's objective is to learn a policy inducing high entropy over states visitation (…
Test-Time Regret Minimization in Meta Reinforcement Learning
Mirco Mutti, Aviv Tamar
Meta reinforcement learning sets a distribution over a set of tasks on which the agent can train at will, then is asked to learn an optimal policy for any test task efficiently. In…
How does Inverse RL Scale to Large State Spaces? A Provably Efficient Approach
Filippo Lazzati, Mirco Mutti, Alberto Maria Metelli
In online Inverse Reinforcement Learning (IRL), the learner can collect samples about the dynamics of the environment to improve its estimate of the reward function. Since IRL suff…
Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms
Filippo Lazzati, Mirco Mutti, Alberto Maria Metelli
Inverse reinforcement learning (IRL) aims to recover the reward function of an expert agent from demonstrations of behavior. It is well-known that the IRL problem is fundamentally…
A Theoretical Framework for Partially Observed Reward-States in RLHF
Chinmaya Kausik, Mirco Mutti, Aldo Pacchiano +1
The growing deployment of reinforcement learning from human feedback (RLHF) calls for a deeper theoretical investigation of its underlying models. The prevalent models of RLHF do n…
Exploiting Causal Graph Priors with Posterior Sampling for Reinforcement Learning
Mirco Mutti, Riccardo De Santi, Marcello Restelli +2
Posterior sampling allows exploitation of prior knowledge on the environment's transition dynamics to improve the sample efficiency of reinforcement learning. The prior is typicall…
Persuading Farsighted Receivers in MDPs: the Power of Honesty
Martino Bernasconi, Matteo Castiglioni, Alberto Marchesi +1
Bayesian persuasion studies the problem faced by an informed sender who strategically discloses information to influence the behavior of an uninformed receiver. Recently, a growing…
A Tale of Sampling and Estimation in Discounted Reinforcement Learning
Alberto Maria Metelli, Mirco Mutti, Marcello Restelli
The most relevant problems in discounted reinforcement learning involve estimating the mean of a function under the stationary distribution of a Markov reward process, such as the…
Reward-Free Policy Space Compression for Reinforcement Learning
Mirco Mutti, Stefano Del Col, Marcello Restelli
In reinforcement learning, we encode the potential behaviors of an agent interacting with an environment into an infinite set of policies, the policy space, typically represented b…
Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization
Mirco Mutti, Riccardo De Santi, Emanuele Rossi +3
In the sequential decision making setting, an agent aims to achieve systematic generalization over a large, possibly infinite, set of environments. Such environments are modeled as…
The Importance of Non-Markovianity in Maximum State Entropy Exploration
Mirco Mutti, Riccardo De Santi, Marcello Restelli
In the maximum state entropy exploration framework, an agent interacts with a reward-free environment to learn a policy that maximizes the entropy of the expected state visitations…
Challenging Common Assumptions in Convex Reinforcement Learning
Mirco Mutti, Riccardo De Santi, Piersilvio De Bartolomeis +1
The classic Reinforcement Learning (RL) formulation concerns the maximization of a scalar reward function. More recently, convex RL has been introduced to extend the RL formulation…