19 citations · 66 across the 26 of their papers we have counts for
29 papers
m-Set Adversarial Bandits with Winner Feedback
Nicolò Cesa-Bianchi, Matteo Papini
We show upper and lower bounds on the regret of -set adversarial bandits for different utilities (winner reward or sum of rewards) and feedback models (winner index, winner rewa…
Sample complexity of variance-reduced policy gradient: weaker assumptions and lower bounds
Gabor Paczolay, Matteo Papini, Alberto Maria Metelli +2
Several variance-reduced versions of REINFORCE based on importance sampling achieve an improved sample complexity to find an -stationary point, under an unrealistic…
Reusing Past Samples in Proximal Policy Optimization: When and How Does It Help?
Alessandro Montenegro, Riccardo Venturelli, Marco Mussi +2
Among on-policy deep reinforcement learning methods, Proximal Policy Optimization (PPO) has become the de facto standard, due to its consistently strong empirical performance acros…
Actor-Critic with Active Importance Sampling
Majid Molaei, Gabor Paczolay, Matteo Papini +2
This paper introduces the Active-Importance-Sampling Actor-Critic (AISAC) algorithm, an extension of the Actor-Critic framework for reducing variance in policy gradient estimation.…
How Log-Barrier Helps Exploration in Policy Optimization
Leonardo Cesani, Matteo Papini, Marcello Restelli
Recently, it has been shown that the Stochastic Gradient Bandit (SGB) algorithm converges to a globally optimal policy with a constant learning rate. However, these guarantees rely…
Impact of Connectivity on Laplacian Representations in Reinforcement Learning
Tommaso Giorgi, Pierriccardo Olivieri, Keyue Jiang +2
Learning compact state representations in Markov Decision Processes (MDPs) has proven crucial for addressing the curse of dimensionality in large-scale reinforcement learning (RL)…
Gym4ReaL: A Suite for Benchmarking Real-World Reinforcement Learning
Davide Salaorni, Vincenzo De Paola, Samuele Delpero +9
In recent years, \emph{Reinforcement Learning} (RL) has made remarkable progress, achieving superhuman performance in a wide range of simulated environments. As research moves towa…
Learning Deterministic Policies with Policy Gradients in Constrained Markov Decision Processes
Alessandro Montenegro, Leonardo Cesani, Marco Mussi +2
Constrained Reinforcement Learning (CRL) addresses sequential decision-making problems where agents are required to achieve goals by maximizing the expected return while meeting do…
Reusing Trajectories in Policy Gradients Enables Fast Convergence
Alessandro Montenegro, Federico Mansutti, Marco Mussi +2
Policy gradient (PG) methods are a class of effective reinforcement learning algorithms, particularly when dealing with continuous control problems. They rely on fresh on-policy da…
Statistical Analysis of Policy Space Compression Problem
Majid Molaei, Marcello Restelli, Alberto Maria Metelli +1
Policy search methods are crucial in reinforcement learning, offering a framework to address continuous state-action and partially observable problems. However, the complexity of e…
Local Linearity: the Key for No-regret Reinforcement Learning in Continuous MDPs
Davide Maran, Alberto Maria Metelli, Matteo Papini +1
Achieving the no-regret property for Reinforcement Learning (RL) problems in continuous state and action-space environments is one of the major open problems in the field. Existing…
Last-Iterate Global Convergence of Policy Gradients for Constrained Reinforcement Learning
Alessandro Montenegro, Marco Mussi, Matteo Papini +1
Constrained Reinforcement Learning (CRL) tackles sequential decision-making problems where agents are required to achieve goals by maximizing the expected return while meeting doma…
Learning Optimal Deterministic Policies with Stochastic Policy Gradients
Alessandro Montenegro, Marco Mussi, Alberto Maria Metelli +1
Policy gradient (PG) methods are successful approaches to deal with continuous reinforcement learning (RL) problems. They learn stochastic parametric (hyper)policies by either expl…
Projection by Convolution: Optimal Sample Complexity for Reinforcement Learning in Continuous-Space MDPs
Davide Maran, Alberto Maria Metelli, Matteo Papini +1
We consider the problem of learning an -optimal policy in a general class of continuous-space Markov decision processes (MDPs) having smooth Bellman operators. Given a…
Policy Gradient with Active Importance Sampling
Matteo Papini, Giorgio Manganini, Alberto Maria Metelli +1
Importance sampling (IS) represents a fundamental technique for a large surge of off-policy reinforcement learning approaches. Policy gradient (PG) methods, in particular, signific…
No-Regret Reinforcement Learning in Smooth MDPs
Davide Maran, Alberto Maria Metelli, Matteo Papini +1
Obtaining no-regret guarantees for reinforcement learning (RL) in the case of problems with continuous state and/or action spaces is still one of the major open challenges in the f…
Importance-Weighted Offline Learning Done Right
Germano Gabbianelli, Gergely Neu, Matteo Papini
We study the problem of offline policy optimization in stochastic contextual bandit problems, where the goal is to learn a near-optimal policy based on a dataset of decision data c…
Offline Primal-Dual Reinforcement Learning for Linear MDPs
Germano Gabbianelli, Gergely Neu, Nneka Okolo +1
Offline Reinforcement Learning (RL) aims to learn a near-optimal policy from a fixed dataset of transitions collected by another policy. This problem has attracted a lot of attenti…
Scalable Representation Learning in Linear Contextual Bandits with Constant Regret Guarantees
Andrea Tirinzoni, Matteo Papini, Ahmed Touati +2
We study the problem of representation learning in stochastic contextual linear bandits. While the primary concern in this domain is usually to find realizable representations (i.e…
Online Learning with Off-Policy Feedback
Germano Gabbianelli, Matteo Papini, Gergely Neu
We study the problem of online learning in adversarial bandit problems under a partial observability model called off-policy feedback. In this sequential decision making problem, t…
Lifting the Information Ratio: An Information-Theoretic Analysis of Thompson Sampling for Contextual Bandits
Gergely Neu, Julia Olkhovskaya, Matteo Papini +1
We study the Bayesian regret of the renowned Thompson Sampling algorithm in contextual bandits with binary losses and adversarially-selected contexts. We adapt the information-theo…
Reinforcement Learning in Linear MDPs: Constant Regret and Representation Selection
Matteo Papini, Andrea Tirinzoni, Aldo Pacchiano +3
We study the role of the representation of state-action value functions in regret minimization in finite-horizon Markov Decision Processes (MDPs) with linear structure. We first de…
Leveraging Good Representations in Linear Contextual Bandits
Matteo Papini, Andrea Tirinzoni, Marcello Restelli +2
The linear contextual bandit literature is mostly focused on the design of efficient learning algorithms for a given representation. However, a contextual bandit problem may admit…
Policy Optimization as Online Learning with Mediator Feedback
Alberto Maria Metelli, Matteo Papini, Pierluca D'Oro +1
Policy Optimization (PO) is a widely used approach to address continuous control tasks. In this paper, we introduce the notion of mediator feedback that frames PO as an online lear…