65 citations · 367 across the 30 of their papers we have counts for
31 papers
Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitation
Yihong Guo, Yixuan Wang, Yuanyuan Shi +2
Training a policy in a source domain for deployment in the target domain under a dynamics shift can be challenging, often resulting in performance degradation. Previous work tackle…
Robust Offline Reinforcement Learning with Linearly Structured f-Divergence Regularization
Cheng Tang, Zhishuai Liu, Pan Xu
The Robust Regularized Markov Decision Process (RRMDP) is proposed to learn policies robust to dynamics shifts by adding regularization to the transition dynamics in the value func…
More Efficient Randomized Exploration for Reinforcement Learning via Approximate Sampling
Haque Ishfaq, Yixin Tan, Yu Yang +5
Thompson sampling (TS) is one of the most popular exploration techniques in reinforcement learning (RL). However, most TS algorithms with theoretical guarantees are difficult to im…
Optimal Batched Linear Bandits
Xuanfei Ren, Tianyuan Jin, Pan Xu
We introduce the E algorithm for the batched linear bandit problem, incorporating an Explore-Estimate-Eliminate-Exploit framework. With a proper choice of exploration rate, we…
Randomized Exploration in Cooperative Multi-Agent Reinforcement Learning
Hao-Lun Hsu, Weixin Wang, Miroslav Pajic +1
We present the first study on provably efficient randomized exploration in cooperative multi-agent reinforcement learning (MARL). We propose a unified algorithm framework for rando…
Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning
Zhishuai Liu, Pan Xu
Distributionally robust offline reinforcement learning (RL), which seeks robust policy training against environment perturbation by modeling dynamics uncertainty, calls for functio…
Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximation
Zhishuai Liu, Pan Xu
We study off-dynamics Reinforcement Learning (RL), where the policy is trained on a source domain and deployed to a distinct target domain. We aim to solve this problem via online…
Optimal Batched Best Arm Identification
Tianyuan Jin, Yu Yang, Jing Tang +2
We study the batched best arm identification (BBAI) problem, where the learner's goal is to identify the best arm while switching the policy as less as possible. In particular, we…
Wasserstein Distributionally Robust Policy Evaluation and Learning for Contextual Bandits
Yi Shen, Pan Xu, Michael M. Zavlanos
Off-policy evaluation and learning are concerned with assessing a given policy and learning an optimal policy from offline data without direct interaction with the environment. Oft…
Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo
Haque Ishfaq, Qingfeng Lan, Pan Xu +4
We present a scalable and effective exploration strategy based on Thompson sampling for reinforcement learning (RL). One of the key shortcomings of existing Thompson sampling algor…
Queer In AI: A Case Study in Community-Led Participatory AI
Organizers Of QueerInAI, :, Anaelia Ovalle +48
We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community…
Langevin Monte Carlo for Contextual Bandits
Pan Xu, Hongkai Zheng, Eric Mazumdar +2
We study the efficiency of Thompson sampling for contextual bandits. Existing Thompson sampling-based algorithms need to construct a Laplace approximation (i.e., a Gaussian distrib…
Finite-Time Regret of Thompson Sampling Algorithms for Exponential Family Multi-Armed Bandits
Tianyuan Jin, Pan Xu, Xiaokui Xiao +1
We study the regret of Thompson sampling (TS) algorithms for exponential family bandits, where the reward distribution is from a one-dimensional exponential family, which covers ma…
Neural Contextual Bandits with Deep Representation and Shallow Exploration
Pan Xu, Zheng Wen, Handong Zhao +1
We study a general class of contextual bandits, where each context-action pair is associated with a raw feature vector, but the reward generating function is unknown. We propose a…
Faster Convergence of Stochastic Gradient Langevin Dynamics for Non-Log-Concave Sampling
Difan Zou, Pan Xu, Quanquan Gu
We provide a new convergence analysis of stochastic gradient Langevin dynamics (SGLD) for sampling from a class of distributions that can be non-log-concave. At the core of our app…
A Finite Time Analysis of Two Time-Scale Actor Critic Methods
Yue Wu, Weitong Zhang, Pan Xu +1
Actor-critic (AC) methods have exhibited great empirical success compared with other reinforcement learning algorithms, where the actor uses the policy gradient to improve the lear…
MOTS: Minimax Optimal Thompson Sampling
Tianyuan Jin, Pan Xu, Jieming Shi +2
Thompson sampling is one of the most widely used algorithms for many online decision problems, due to its simplicity in implementation and superior empirical performance over other…
Double Explore-then-Commit: Asymptotic Optimality and Beyond
Tianyuan Jin, Pan Xu, Xiaokui Xiao +1
We study the multi-armed bandit problem with subgaussian rewards. The explore-then-commit (ETC) strategy, which consists of an exploration phase followed by an exploitation phase,…
Rank Aggregation via Heterogeneous Thurstone Preference Models
Tao Jin, Pan Xu, Quanquan Gu +1
We propose the Heterogeneous Thurstone Model (HTM) for aggregating ranked data, which can take the accuracy levels of different users into account. By allowing different noise dist…
A Finite-Time Analysis of Q-Learning with Neural Network Function Approximation
Pan Xu, Quanquan Gu
Q-learning with neural network function approximation (neural Q-learning for short) is among the most prevalent deep reinforcement learning algorithms. Despite its empirical succes…
Sample Efficient Policy Gradient Methods with Recursive Variance Reduction
Pan Xu, Felicia Gao, Quanquan Gu
Improving the sample efficiency in reinforcement learning has been a long-standing research problem. In this work, we aim to reduce the sample complexity of existing policy gradien…
An Improved Convergence Analysis of Stochastic Variance-Reduced Policy Gradient
Pan Xu, Felicia Gao, Quanquan Gu
We revisit the stochastic variance-reduced policy gradient (SVRPG) method proposed by Papini et al. (2018) for reinforcement learning. We provide an improved convergence analysis o…
Sample Efficient Stochastic Variance-Reduced Cubic Regularization Method
Dongruo Zhou, Pan Xu, Quanquan Gu
We propose a sample efficient stochastic variance-reduced cubic regularization (Lite-SVRC) algorithm for finding the local minimum efficiently in nonconvex optimization. The propos…
Finding Local Minima via Stochastic Nested Variance Reduction
Dongruo Zhou, Pan Xu, Quanquan Gu
We propose two algorithms that can find local minima faster than the state-of-the-art algorithms in both finite-sum and general stochastic nonconvex optimization. At the core of th…