11 papers
Refined Analysis of Entropy-Regularized Actor-Critic
Safwan Labbi, Paul Mangold, Daniil Tiapkin +1
In this paper, we study the role of the critic in actor--critic for entropy-regularized, finite, discounted environments. We establish that, when the critic is exact, using the lat…
Gaussian Approximation and Multiplier Bootstrap for Federated Linear Stochastic Approximation
Ilya Levin, Maksim Shuklin, Eric Moulines +2
In this paper, we establish Berry-Esseen-type bounds for federated linear stochastic approximation (LSA). Our results provide the first federated Gaussian approximations for LSA th…
Beyond Softmax and Entropy: Convergence Rates of Policy Gradients with f-SoftArgmax Parameterization & Coupled Regularization
Safwan Labbi, Daniil Tiapkin, Paul Mangold +1
Policy gradient methods are known to be highly sensitive to the choice of policy parameterization. In particular, the widely used softmax parameterization can induce ill-conditione…
On Global Convergence Rates for Federated Softmax Policy Gradient under Heterogeneous Environments
Safwan Labbi, Paul Mangold, Daniil Tiapkin +1
We provide global convergence rates for vanilla and entropy-regularized federated softmax stochastic policy gradient (FedPG) with local training. We show that FedPG converges to a…
Convergence Guarantees for Federated SARSA with Local Training and Heterogeneous Agents
Paul Mangold, Eloïse Berthier, Eric Moulines
We present a novel theoretical analysis of Federated SARSA (FedSARSA) with linear function approximation and local training. We establish convergence guarantees for FedSARSA in the…
Tight Analysis of Decentralized SGD: A Markov Chain Perspective
Lucas Versini, Paul Mangold, Aymeric Dieuleveut
We propose a novel analysis of the Decentralized Stochastic Gradient Descent (DSGD) algorithm with constant step size, interpreting the iterates of the algorithm as a Markov chain.…