6 citations · 8 across the 17 of their papers we have counts for
18 papers
Q-Learning with Stable Infinite-Dimensional Linear Function Approximation
Shengbo Wang
Q-learning with linear function approximation can be unstable because an arbitrary approximation architecture need not preserve the Bellman contraction. We develop a stable infinit…
Lipschitz Regularity in Wasserstein Robust Stochastic Optimal Control
Shengbo Wang, Jose Blanchet
Robust Markov decision processes provide a principled framework for protecting sequential decision-making against transition-law misspecification and have attracted substantial rec…
Fast Convergence of Policy Regret in Learning Stochastic Optimal Control
Shengbo Wang, Jose Blanchet, Peter Glynn
Policy learning in modern operations environments faces a fundamental tension between limited operational data and the large, often continuous, state and action spaces over which g…
Central Limit Theorem for Two-Time-Scale Approximate Distributionally Robust RL
Shengbo Wang, Zexi Zhang
Designing model-free algorithms for distributionally robust reinforcement learning (DRRL) poses fundamental challenges. The robust Bellman operator is nonlinear in the transition k…
Non-Rectangular Average-Reward Robust MDPs: Optimal Policies and Their Transient Values
Shengbo Wang, Nian Si
We study non-rectangular robust Markov decision processes under the average-reward criterion, where the ambiguity set couples transition probabilities across states and the adversa…
Q-Measure-Learning for Continuous State RL: Efficient Implementation and Convergence
Shengbo Wang
We study reinforcement learning in infinite-horizon discounted Markov decision processes with continuous state spaces, where data are generated online from a single trajectory unde…
Achieving Dependence for Average-Reward Q-Learning with a New Contraction Principle
Zijun Chen, Zaiwei Chen, Nian Si +1
We present the convergence rates of synchronous and asynchronous Q-learning for average-reward Markov decision processes, where the absence of contraction poses a fundamental chall…
Bellman Optimality of Average-Reward Robust Markov Decision Processes with a Constant Gain
Shengbo Wang, Nian Si
Learning and optimal control under robust Markov decision processes (MDPs) have received increasing attention, yet most existing theory, algorithms, and applications focus on finit…
Near-Optimal Sample Complexities of Divergence-based S-rectangular Distributionally Robust Reinforcement Learning
Zhenghao Li, Shengbo Wang, Nian Si
Distributionally robust reinforcement learning (DR-RL) has recently gained significant attention as a principled approach that addresses discrepancies between training and testing…
Sample Complexity of Distributionally Robust Average-Reward Reinforcement Learning
Zijun Chen, Shengbo Wang, Nian Si
Motivated by practical applications where stable long-term performance is critical-such as robotics, operations research, and healthcare-we study the problem of distributionally ro…
Dynamic Programming-Compatible Uncertainty Sets in Robust Markov Decision Processes
Julien Grand-Clément, Nian Si, Shengbo Wang
In this paper, we investigate the compatibility of robust Markov Decision Processes (RMDPs) with dynamic programming under various assumptions on the uncertainty set, i.e., we inve…
An Efficient High-Dimensional Gradient Estimator for Stochastic Differential Equations
Shengbo Wang, Jose Blanchet, Peter Glynn
Overparameterized stochastic differential equation (SDE) models have achieved remarkable success in various complex environments, such as PDE-constrained optimization, stochastic c…
Learning Optimal Distributionally Robust Stochastic Control in Continuous State Spaces
Shengbo Wang, Jason Meng, Nian Si +2
We study data-driven learning of robust stochastic control for infinite-horizon systems with potentially continuous state and action spaces. In many managerial settings--supply cha…
On the Foundation of Distributionally Robust Reinforcement Learning
Shengbo Wang, Nian Si, Jose Blanchet +1
Motivated by the need for a robust policy in the face of environment shifts between training and deployment, we contribute to the theoretical foundation of distributionally robust…
Optimal Sample Complexity for Average Reward Markov Decision Processes
Shengbo Wang, Jose Blanchet, Peter Glynn
We resolve the open question regarding the sample complexity of policy learning for maximizing the long-run average reward associated with a uniformly ergodic Markov decision proce…
Sample Complexity of Variance-reduced Distributionally Robust Q-learning
Shengbo Wang, Nian Si, Jose Blanchet +1
Dynamic decision-making under distributional shifts is of fundamental interest in theory and applications of reinforcement learning: The distribution of the environment in which th…
A Finite Sample Complexity Bound for Distributionally Robust Q-learning
Shengbo Wang, Nian Si, Jose Blanchet +1
We consider a reinforcement learning setting in which the deployment environment is different from the training environment. Applying a robust Markov decision processes formulation…
Optimal Sample Complexity of Stable Discounted Markov Decision Processes
Shengbo Wang, Jose Blanchet, Peter Glynn
We study the optimal sample complexity of tabular reinforcement learning for infinite-horizon discounted Markov decision processes. The unrestricted minimax sample complexity is kn…