5 papers
Efficient Hypergradient Descent for Inverse Reinforcement Learning
Nikita Sevriukov, Anna Barabanova, Uliana Gagarina +4
Inverse reinforcement learning (IRL) aims to recover a reward function under which the resulting policy reproduces the behavior observed in expert demonstrations. A natural approac…
Gaussian Approximation and Multiplier Bootstrap for Federated Linear Stochastic Approximation
Ilya Levin, Maksim Shuklin, Eric Moulines +2
In this paper, we establish Berry-Esseen-type bounds for federated linear stochastic approximation (LSA). Our results provide the first federated Gaussian approximations for LSA th…
UVIP: Model-Free Approach to Evaluate Reinforcement Learning Algorithms
Denis Belomestny, Ilya Levin, Alexey Naumov +1
Policy evaluation is an important instrument for the comparison of different algorithms in Reinforcement Learning (RL). However, even a precise knowledge of the value function $V^Ï…
SCAFFLSA: Taming Heterogeneity in Federated Linear Stochastic Approximation and TD Learning
Paul Mangold, Sergey Samsonov, Safwan Labbi +4
In this paper, we analyze the sample and communication complexity of the federated linear stochastic approximation (FedLSA) algorithm. We explicitly quantify the effects of local t…
High-Order Error Bounds for Markovian LSA with Richardson-Romberg Extrapolation
Ilya Levin, Alexey Naumov, Sergey Samsonov
In this paper, we study the bias and high-order error bounds of the Linear Stochastic Approximation (LSA) algorithm with Polyak-Ruppert (PR) averaging under Markovian noise. We foc…