1 citations · 2 across the 9 of their papers we have counts for
10 papers
Achieving an Optimality Gap in Average-Reward Weakly-Coupled MDPs
Yige Hong, Xiangcheng Zhang, Qiaomin Xie +2
We study average-reward weakly-coupled Markov decision processes (WCMDPs), where a WCMDP consists of smaller MDPs, called arms, that share multiple per-step budget constraints.…
Lyapunov-Based Sample Complexity Analysis for Weakly-Coupled MDPs
Tianhao Wu, Matthew Zurek, Weina Wang +1
We study the sample complexity of learning in average-reward weakly-coupled Markov decision processes (WCMDPs) and Restless Bandits (RBs) under a generative model. Naive reduction…
Wasserstein-p Central Limit Theorem Rates: From Local Dependence to Markov Chains
Yixuan Zhang, Qiaomin Xie
Non-asymptotic central limit theorem (CLT) rates play a central role in modern machine learning and operations research. In this paper, we study CLT rates for multivariate dependen…
Contextual Online Pricing with (Biased) Offline Data
Yixuan Zhang, Ruihao Zhu, Qiaomin Xie
We study contextual online pricing with biased offline data. For the scalar price elasticity case, we identify the instance-dependent quantity that measures how far the offli…
A Piecewise Lyapunov Analysis of Sub-quadratic SGD: Applications to Robust and Quantile Regression
Yixuan Zhang, Dongyan Huo, Yudong Chen +1
Motivated by robust and quantile regression problems, we investigate the stochastic gradient descent (SGD) algorithm for minimizing an objective function that is locally strong…
Two-Timescale Linear Stochastic Approximation: Constant Stepsizes Go a Long Way
Jeongyeol Kwon, Luke Dotson, Yudong Chen +1
Previous studies on two-timescale stochastic approximation (SA) mainly focused on bounding mean-squared errors under diminishing stepsize schemes. In this work, we investigate {\it…
The Collusion of Memory and Nonlinearity in Stochastic Approximation With Constant Stepsize
Dongyan Huo, Yixuan Zhang, Yudong Chen +1
In this work, we investigate stochastic approximation (SA) with Markovian data and nonlinear updates under constant stepsize . Existing work has primarily focused on either i.…
Achieving Exponential Asymptotic Optimality in Average-Reward Restless Bandits without Global Attractor Assumption
Yige Hong, Qiaomin Xie, Yudong Chen +1
We consider the infinite-horizon average-reward restless bandit problem. We propose a novel two-set policy that maintains two dynamic subsets of arms: one subset of arms has a near…
Prelimit Coupling and Steady-State Convergence of Constant-stepsize Nonsmooth Contractive SA
Yixuan Zhang, Dongyan Huo, Yudong Chen +1
Motivated by Q-learning, we study nonsmooth contractive stochastic approximation (SA) with constant stepsize. We focus on two important classes of dynamics: 1) nonsmooth contractiv…
Constant Stepsize Q-learning: Distributional Convergence, Bias and Extrapolation
Yixuan Zhang, Qiaomin Xie
Stochastic Approximation (SA) is a widely used algorithmic approach in various fields, including optimization and reinforcement learning (RL). Among RL algorithms, Q-learning is pa…