11 papers · 1 filter
Finite-Time Analysis of Discounted Exponential-Utility Reinforcement Learning
Ankur Naskar, Vivek T A, Aditya Kumar +2
Discounted exponential utility provides a principled criterion for risk-sensitive sequential decision-making, but its nonlinear structure complicates reinforcement learning. A rece…
Adversary-Robust Learning from Fully Asynchronous Directional Derivative Estimates
Anik Kumar Paul, Nibedita Roy, Nagesh Talagani +3
We propose FAR-SIGN (Fully Asynchronous Robust optimization via SIGNed directional projections) for adversary-resilient learning in parameter-server--worker systems. FAR-SIGN achie…
Reinforcement Learning for Exponential Utility: Algorithms and Convergence in Discounted MDPs
Gugan Thoppe, L. A. Prashanth, Ankur Naskar +1
Reinforcement learning (RL) for exponential-utility optimization in discounted Markov decision processes (MDPs) lacks principled value-based algorithms. We address this gap in the…
Monotone and Conservative Policy Iteration Beyond the Tabular Case
S. R. Eshwar, Gugan Thoppe, Ananyabrata Barua +2
We introduce Reliable Policy Iteration (RPI) and Conservative RPI (CRPI), variants of Policy Iteration (PI) and Conservative PI (CPI), that retain tabular guarantees under function…
Parameter-free Optimal Rates for Nonlinear Semi-Norm Contractions with Applications to -Learning
Ankur Naskar, Gugan Thoppe, Vijay Gupta
Algorithms for solving \textit{nonlinear} fixed-point equations -- such as average-reward \textit{-learning} and \textit{TD-learning} -- often involve semi-norm contractions. Ac…
Parameter-Free Federated TD Learning with Markov Noise in Heterogeneous Environments
Ankur Naskar, Gugan Thoppe, Utsav Negi +1
Federated learning (FL) can dramatically speed up reinforcement learning by distributing exploration and training across multiple agents. It can guarantee an optimal convergence ra…