19 citations · 40 across the 20 of their papers we have counts for
24 papers
Reusing Past Samples in Proximal Policy Optimization: When and How Does It Help?
Alessandro Montenegro, Riccardo Venturelli, Marco Mussi +2
Among on-policy deep reinforcement learning methods, Proximal Policy Optimization (PPO) has become the de facto standard, due to its consistently strong empirical performance acros…
Why Does Action Chunking Improve Behavioral Cloning Performance in Robotic Control?
Filippo Lazzati, Kyle Stachowicz, William Chen +3
Action chunking---predicting and executing multiple actions instead of a single action---has proven to be a critical component for learning effective robotic control policies. Howe…
On Generalizing Preference-based Reinforcement Learning to Multi-Objective Problems
Simone Drago, Marco Mussi, Leonardo Bianconi +1
We study reinforcement learning (RL) from pairwise human-generated trajectory comparisons. We generalize preference-based RL by formalizing a novel setting in which the expert can…
Mind Your Steps: A General Learning Framework for Accurate Humanoid Foothold Tracking
Alessandro Montenegro, Shihao Li, Puze Liu +2
Enabling humanoid robots to operate in complex, dynamic environments remains a critical challenge, fundamentally limited by the ability to navigate robustly, safely, and accurately…
Imitation Learning as Return Distribution Matching
Filippo Lazzati, Alberto Maria Metelli
We study the problem of training a risk-sensitive reinforcement learning (RL) agent through imitation learning (IL). Unlike standard IL, our goal is not only to train an agent that…
Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
Filippo Lazzati, Alberto Maria Metelli
We study the problem of generalizing an expert agent's behavior, provided through demonstrations, to new environments and/or additional constraints. Inverse Reinforcement Learning…
Learning Deterministic Policies with Policy Gradients in Constrained Markov Decision Processes
Alessandro Montenegro, Leonardo Cesani, Marco Mussi +2
Constrained Reinforcement Learning (CRL) addresses sequential decision-making problems where agents are required to achieve goals by maximizing the expected return while meeting do…
Reusing Trajectories in Policy Gradients Enables Fast Convergence
Alessandro Montenegro, Federico Mansutti, Marco Mussi +2
Policy gradient (PG) methods are a class of effective reinforcement learning algorithms, particularly when dealing with continuous control problems. They rely on fresh on-policy da…
Reward Compatibility: A Framework for Inverse RL
Filippo Lazzati, Mirco Mutti, Alberto Metelli
We provide an original theoretical study of Inverse Reinforcement Learning (IRL) through the lens of reward compatibility, a novel framework to quantify the compatibility of a rewa…
Robustness in the Face of Partial Identifiability in Reward Learning
Filippo Lazzati, Alberto Maria Metelli
In Reward Learning (ReL), we are given feedback on an unknown target reward, and the goal is to use this information to recover it in order to carry out some downstream application…
Learning Utilities from Demonstrations in Markov Decision Processes
Filippo Lazzati, Alberto Maria Metelli
Our goal is to extract useful knowledge from demonstrations of behavior in sequential decision-making problems. Although it is well-known that humans commonly engage in risk-sensit…
Last-Iterate Global Convergence of Policy Gradients for Constrained Reinforcement Learning
Alessandro Montenegro, Marco Mussi, Matteo Papini +1
Constrained Reinforcement Learning (CRL) tackles sequential decision-making problems where agents are required to achieve goals by maximizing the expected return while meeting doma…
How does Inverse RL Scale to Large State Spaces? A Provably Efficient Approach
Filippo Lazzati, Mirco Mutti, Alberto Maria Metelli
In online Inverse Reinforcement Learning (IRL), the learner can collect samples about the dynamics of the environment to improve its estimate of the reward function. Since IRL suff…
Learning Optimal Deterministic Policies with Stochastic Policy Gradients
Alessandro Montenegro, Marco Mussi, Alberto Maria Metelli +1
Policy gradient (PG) methods are successful approaches to deal with continuous reinforcement learning (RL) problems. They learn stochastic parametric (hyper)policies by either expl…
Performance Improvement Bounds for Lipschitz Configurable Markov Decision Processes
Alberto Maria Metelli
Configurable Markov Decision Processes (Conf-MDPs) have recently been introduced as an extension of the traditional Markov Decision Processes (MDPs) to model the real-world scenari…
No-Regret Reinforcement Learning in Smooth MDPs
Davide Maran, Alberto Maria Metelli, Matteo Papini +1
Obtaining no-regret guarantees for reinforcement learning (RL) in the case of problems with continuous state and/or action spaces is still one of the major open challenges in the f…
Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms
Filippo Lazzati, Mirco Mutti, Alberto Maria Metelli
Inverse reinforcement learning (IRL) aims to recover the reward function of an expert agent from demonstrations of behavior. It is well-known that the IRL problem is fundamentally…
Policy Optimization as Online Learning with Mediator Feedback
Alberto Maria Metelli, Matteo Papini, Pierluca D'Oro +1
Policy Optimization (PO) is a widely used approach to address continuous control tasks. In this paper, we introduce the notion of mediator feedback that frames PO as an online lear…
Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learning
Alberto Maria Metelli, Flavio Mazzolini, Lorenzo Bisi +2
The choice of the control frequency of a system has a relevant impact on the ability of reinforcement learning algorithms to learn a highly performing policy. In this paper, we int…
Policy Space Identification in Configurable Environments
Alberto Maria Metelli, Guglielmo Manneschi, Marcello Restelli
We study the problem of identifying the policy space of a learning agent, having access to a set of demonstrations generated by its optimal policy. We introduce an approach based o…
Gradient-Aware Model-based Policy Search
Pierluca D'Oro, Alberto Maria Metelli, Andrea Tirinzoni +2
Traditional model-based reinforcement learning approaches learn a model of the environment dynamics without explicitly considering how it will be used by the agent. In the presence…
Feature Selection via Mutual Information: New Theoretical Insights
Mario Beraha, Alberto Maria Metelli, Matteo Papini +2
Mutual information has been successfully adopted in filter feature-selection methods to assess both the relevancy of a subset of features in predicting the target variable and the…
Policy Optimization via Importance Sampling
Alberto Maria Metelli, Matteo Papini, Francesco Faccio +1
Policy optimization is an effective reinforcement learning approach to solve continuous control tasks. Recent achievements have shown that alternating online and offline optimizati…
Configurable Markov Decision Processes
Alberto Maria Metelli, Mirco Mutti, Marcello Restelli
In many real-world problems, there is the possibility to configure, to a limited extent, some environmental parameters to improve the performance of a learning agent. In this paper…