528 citations · 872 across the 40 of their papers we have counts for
43 papers
Sample Complexity of Variance-reduced Distributionally Robust Q-learning
Shengbo Wang, Nian Si, Jose Blanchet +1
Dynamic decision-making under distributional shifts is of fundamental interest in theory and applications of reinforcement learning: The distribution of the environment in which th…
Stochastic Nonsmooth Convex Optimization with Heavy-Tailed Noises: High-Probability Bound, In-Expectation Rate and Initial Distance Adaptation
Zijian Liu, Zhengyuan Zhou
Recently, several studies consider the stochastic optimization problem but in a heavy-tailed noise regime, i.e., the difference between the stochastic gradient and the true gradien…
A Finite Sample Complexity Bound for Distributionally Robust Q-learning
Shengbo Wang, Nian Si, Jose Blanchet +1
We consider a reinforcement learning setting in which the deployment environment is different from the training environment. Applying a robust Markov decision processes formulation…
Breaking the Lower Bound with (Little) Structure: Acceleration in Non-Convex Stochastic Optimization with Heavy-Tailed Noise
Zijian Liu, Jiawei Zhang, Zhengyuan Zhou
We consider the stochastic optimization problem with smooth but not necessarily convex objectives in the heavy-tailed noise regime, where the stochastic gradient's noise is assumed…
Near-Optimal Non-Convex Stochastic Optimization under Generalized Smoothness
Zijian Liu, Srikanth Jagabathula, Zhengyuan Zhou
The generalized smooth condition, -smoothness, has triggered people's interest since it is more realistic in many optimization problems shown by both empirical and t…
Single-Trajectory Distributionally Robust Reinforcement Learning
Zhipeng Liang, Xiaoteng Ma, Jose Blanchet +2
To mitigate the limitation that the classical reinforcement learning (RL) framework heavily relies on identical training and test environments, Distributionally Robust RL (DRRL) ha…
Leveraging the Hints: Adaptive Bidding in Repeated First-Price Auctions
Wei Zhang, Yanjun Han, Zhengyuan Zhou +2
With the advent and increasing consolidation of e-commerce, digital advertising has very recently replaced traditional advertising as the main marketing force in the economy. In th…
Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation
Xiaoteng Ma, Zhipeng Liang, Jose Blanchet +5
Among the reasons hindering reinforcement learning (RL) applications to real-world problems, two factors are critical: limited data and the mismatch between the testing environment…
Optimal Diagonal Preconditioning
Zhaonan Qu, Wenzhi Gao, Oliver Hinder +2
Preconditioning has long been a staple technique in optimization, often applied to reduce the condition number of a matrix and speed up the convergence of algorithms. Although ther…
Learning to Order for Inventory Systems with Lost Sales and Uncertain Supplies
Boxiao Chen, Jiashuo Jiang, Jiawei Zhang +1
We consider a stochastic lost-sales inventory control system with a lead time over a planning horizon . Supply is uncertain, and is a function of the order quantity (due to…
Sensitivity analysis under the -sensitivity models: a distributional robustness perspective
Ying Jin, Zhimei Ren, Zhengyuan Zhou
This paper introduces the -sensitivity model, a new sensitivity model that characterizes the violation of unconfoundedness in causal inference. It assumes the selection bias due…
Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning
Nathan Kallus, Xiaojie Mao, Kaiwen Wang +1
Off-policy evaluation and learning (OPE/L) use offline observational data to make better decisions, which is crucial in applications where online experimentation is limited. Howeve…
Doubly Optimal No-Regret Online Learning in Strongly Monotone Games with Bandit Feedback
Wenjia Ba, Tianyi Lin, Jiawei Zhang +1
We consider online no-regret learning in unknown games with bandit feedback, where each player can only observe its reward at each time -- determined by all players' current joint…
Distributed stochastic optimization with large delays
Zhengyuan Zhou, Panayotis Mertikopoulos, Nicholas Bambos +2
One of the most widely used methods for solving large-scale stochastic optimization problems is distributed asynchronous stochastic gradient descent (DASGD), a family of algorithms…
Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learning
Yuexiang Zhai, Christina Baek, Zhengyuan Zhou +2
Many goal-reaching reinforcement learning (RL) tasks have empirically verified that rewarding the agent on subgoals improves convergence speed and practical performance. We attempt…
Policy Learning with Adaptively Collected Data
Ruohan Zhan, Zhimei Ren, Susan Athey +1
Learning optimal policies from historical data enables personalization in a wide variety of applications including healthcare, digital recommendations, and online education. The gr…
Optimistic Dual Extrapolation for Coherent Non-monotone Variational Inequalities
Chaobing Song, Zhengyuan Zhou, Yichao Zhou +2
The optimization problems associated with training generative adversarial neural networks can be largely reduced to certain {\em non-monotone} variational inequality problems (VIPs…
No Weighted-Regret Learning in Adversarial Bandits with Delays
Ilai Bistritz, Zhengyuan Zhou, Xi Chen +2
Consider a scenario where a player chooses an action in each round out of rounds and observes the incurred cost after a delay of rounds. The cost functions and the…
Online Multi-Armed Bandits with Adaptive Inference
Maria Dimakopoulou, Zhimei Ren, Zhengyuan Zhou
During online decision making in Multi-Armed Bandits (MAB), one needs to conduct inference on the true mean reward of each arm based on data collected so far at each step. However,…
Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent States
Shi Dong, Benjamin Van Roy, Zhengyuan Zhou
We design a simple reinforcement learning (RL) agent that implements an optimistic version of -learning and establish through regret analysis that this agent can operate with so…
Comments on Efficient Singular Value Thresholding Computation
Zhengyuan Zhou, Yi Ma
We discuss how to evaluate the proximal operator of a convex and increasing function of a nuclear norm, which forms the key computational step in several first-order optimization a…
LQR with Tracking: A Zeroth-order Approach and Its Global Convergence
Zhaolin Ren, Aoxiao Zhong, Na Li
There has been substantial recent progress on the theoretical understanding of model-free approaches to Linear Quadratic Regulator (LQR) problems. Much attention has been devoted t…
Dynamic Batch Learning in High-Dimensional Sparse Linear Contextual Bandits
Zhimei Ren, Zhengyuan Zhou
We study the problem of dynamic batch learning in high-dimensional sparse linear contextual bandits, where a decision maker, under a given maximum-number-of-batch constraint and on…
A Unified Linear Speedup Analysis of Federated Averaging and Nesterov FedAvg
Zhaonan Qu, Kaixiang Lin, Zhaojian Li +2
Federated learning (FL) learns a model jointly from a set of participating devices without sharing each other's privately held data. The characteristics of non-i.i.d. data across t…