6 papers
Non-Rectangular Average-Reward Robust MDPs: Optimal Policies and Their Transient Values
Shengbo Wang, Nian Si
We study non-rectangular robust Markov decision processes under the average-reward criterion, where the ambiguity set couples transition probabilities across states and the adversa…
Achieving Dependence for Average-Reward Q-Learning with a New Contraction Principle
Zijun Chen, Zaiwei Chen, Nian Si +1
We present the convergence rates of synchronous and asynchronous Q-learning for average-reward Markov decision processes, where the absence of contraction poses a fundamental chall…
Bellman Optimality of Average-Reward Robust Markov Decision Processes with a Constant Gain
Shengbo Wang, Nian Si
Learning and optimal control under robust Markov decision processes (MDPs) have received increasing attention, yet most existing theory, algorithms, and applications focus on finit…
Near-Optimal Sample Complexities of Divergence-based S-rectangular Distributionally Robust Reinforcement Learning
Zhenghao Li, Shengbo Wang, Nian Si
Distributionally robust reinforcement learning (DR-RL) has recently gained significant attention as a principled approach that addresses discrepancies between training and testing…
Sample Complexity of Distributionally Robust Average-Reward Reinforcement Learning
Zijun Chen, Shengbo Wang, Nian Si
Motivated by practical applications where stable long-term performance is critical-such as robotics, operations research, and healthcare-we study the problem of distributionally ro…
Dynamic Programming-Compatible Uncertainty Sets in Robust Markov Decision Processes
Julien Grand-Clément, Nian Si, Shengbo Wang
In this paper, we investigate the compatibility of robust Markov Decision Processes (RMDPs) with dynamic programming under various assumptions on the uncertainty set, i.e., we inve…