61 citations · 255 across the 19 of their papers we have counts for
19 papers
A Simple Algorithm For Scaling Up Kernel Methods
Teng Andrea Xu, Bryan Kelly, Semyon Malamud
The recent discovery of the equivalence between infinitely wide neural networks (NNs) in the lazy training regime and Neural Tangent Kernels (NTKs) (Jacot et al., 2018) has revived…
Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Reward
Tengyu Xu, Yue Wang, Shaofeng Zou +1
The remarkable success of reinforcement learning (RL) heavily relies on observing the reward of every visited state-action pair. In many real world applications, however, an agent…
Model-Based Offline Meta-Reinforcement Learning with Regularization
Sen Lin, Jialin Wan, Tengyu Xu +2
Existing offline reinforcement learning (RL) methods face a few major challenges, particularly the distributional shift between the learned policy and the behavior policy. Offline…
Faster Algorithm and Sharper Analysis for Constrained Markov Decision Process
Tianjiao Li, Ziwei Guan, Shaofeng Zou +3
The problem of constrained Markov decision process (CMDP) is investigated, where an agent aims to maximize the expected accumulated discounted reward subject to multiple constraint…
PER-ETD: A Polynomially Efficient Emphatic Temporal Difference Learning Method
Ziwei Guan, Tengyu Xu, Yingbin Liang
Emphatic temporal difference (ETD) learning (Sutton et al., 2016) is a successful method to conduct the off-policy value function evaluation with function approximation. Although E…
A Unified Off-Policy Evaluation Approach for General Value Function
Tengyu Xu, Zhuoran Yang, Zhaoran Wang +1
General Value Function (GVF) is a powerful tool to represent both the {\em predictive} and {\em retrospective} knowledge in reinforcement learning (RL). In practice, often multiple…
Proximal Gradient Descent-Ascent: Variable Convergence under KŁ Geometry
Ziyi Chen, Yi Zhou, Tengyu Xu +1
The gradient descent-ascent (GDA) algorithm has been widely applied to solve minimax optimization problems. In order to achieve convergent policy parameters for minimax optimizatio…
Doubly Robust Off-Policy Actor-Critic: Convergence and Optimality
Tengyu Xu, Zhuoran Yang, Zhaoran Wang +1
Designing off-policy reinforcement learning algorithms is typically a very challenging task, because a desirable iteration update often involves an expectation over an on-policy di…
Sample Complexity Bounds for Two Timescale Value-based Reinforcement Learning Algorithms
Tengyu Xu, Yingbin Liang
Two timescale stochastic approximation (SA) has been widely used in value-based reinforcement learning algorithms. In the policy evaluation setting, it can model the linear and non…
CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guarantee
Tengyu Xu, Yingbin Liang, Guanghui Lan
In safe reinforcement learning (SRL) problems, an agent explores the environment to maximize an expected total reward and meanwhile avoids violation of certain constraints on a num…
When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence
Ziwei Guan, Tengyu Xu, Yingbin Liang
Generative adversarial imitation learning (GAIL) is a popular inverse reinforcement learning approach for jointly optimizing policy and reward from expert trajectories. A primary q…
Gradient Free Minimax Optimization: Variance Reduction and Faster Convergence
Tengyu Xu, Zhe Wang, Yingbin Liang +1
Many important machine learning applications amount to solving minimax optimization problems, and in many cases there is no access to the gradient information, but only the functio…
Non-asymptotic Convergence Analysis of Two Time-scale (Natural) Actor-Critic Algorithms
Tengyu Xu, Zhe Wang, Yingbin Liang
As an important type of reinforcement learning algorithms, actor-critic (AC) and natural actor-critic (NAC) algorithms are often executed in two ways for finding optimal policies.…
Improving Sample Complexity Bounds for (Natural) Actor-Critic Algorithms
Tengyu Xu, Zhe Wang, Yingbin Liang
The actor-critic (AC) algorithm is a popular method to find an optimal policy in reinforcement learning. In the infinite horizon scenario, the finite-sample convergence rate for th…
Non-asymptotic Convergence of Adam-type Reinforcement Learning Algorithms under Markovian Sampling
Huaqing Xiong, Tengyu Xu, Yingbin Liang +1
Despite the wide applications of Adam in reinforcement learning (RL), the theoretical convergence of Adam-type RL algorithms has not been established. This paper provides the first…
Reanalysis of Variance Reduced Temporal Difference Learning
Tengyu Xu, Zhe Wang, Yi Zhou +1
Temporal difference (TD) learning is a popular algorithm for policy evaluation in reinforcement learning, but the vanilla TD can substantially suffer from the inherent optimization…
Two Time-scale Off-Policy TD Learning: Non-asymptotic Analysis over Markovian Samples
Tengyu Xu, Shaofeng Zou, Yingbin Liang
Gradient-based temporal difference (GTD) algorithms are widely used in off-policy learning scenarios. Among them, the two time-scale TD with gradient correction (TDC) algorithm has…
Finite-Sample Analysis for SARSA with Linear Function Approximation
Shaofeng Zou, Tengyu Xu, Yingbin Liang
SARSA is an on-policy algorithm to learn a Markov decision process policy in reinforcement learning. We investigate the SARSA algorithm with linear function approximation under the…
When Will Gradient Methods Converge to Max-margin Classifier under ReLU Models?
Tengyu Xu, Yi Zhou, Kaiyi Ji +1
We study the implicit bias of gradient descent methods in solving a binary classification problem over a linearly separable dataset. The classifier is described by a nonlinear ReLU…