4 papers
Learning to Control Coupled-Dynamics Environments with Joint Markov Decision Processes
Ege C. Kaya, Aliasghar Pourghani, Mahsa Ghasemi +2
Coupled-dynamics environments expose the one-step outcomes that would follow from several possible counterfactual actions under a common realization of exogenous randomness. The or…
Quotient-Categorical Representations for Bellman-Compatible Average-Reward Distributional Reinforcement Learning
Ege C. Kaya, Aliasghar Pourghani, Vijay Gupta +1
Average-reward reinforcement learning requires estimating the gain and the bias, which is defined only up to an additive constant. This makes direct distributional analogues ill-po…
Parameter-free Optimal Rates for Nonlinear Semi-Norm Contractions with Applications to -Learning
Ankur Naskar, Gugan Thoppe, Vijay Gupta
Algorithms for solving \textit{nonlinear} fixed-point equations -- such as average-reward \textit{-learning} and \textit{TD-learning} -- often involve semi-norm contractions. Ac…
Parameter-Free Federated TD Learning with Markov Noise in Heterogeneous Environments
Ankur Naskar, Gugan Thoppe, Utsav Negi +1
Federated learning (FL) can dramatically speed up reinforcement learning by distributing exploration and training across multiple agents. It can guarantee an optimal convergence ra…