61 citations · 750 across the 66 of their papers we have counts for
8 papers · 1 filter
Faster Algorithm and Sharper Analysis for Constrained Markov Decision Process
Tianjiao Li, Ziwei Guan, Shaofeng Zou +3
The problem of constrained Markov decision process (CMDP) is investigated, where an agent aims to maximize the expected accumulated discounted reward subject to multiple constraint…
On the Convergence Theory for Hessian-Free Bilevel Algorithms
Daouda Sow, Kaiyi Ji, Yingbin Liang
Bilevel optimization has arisen as a powerful tool in modern machine learning. However, due to the nested structure of bilevel optimization, even gradient-based methods require sec…
PER-ETD: A Polynomially Efficient Emphatic Temporal Difference Learning Method
Ziwei Guan, Tengyu Xu, Yingbin Liang
Emphatic temporal difference (ETD) learning (Sutton et al., 2016) is a successful method to conduct the off-policy value function evaluation with function approximation. Although E…
A Unified Off-Policy Evaluation Approach for General Value Function
Tengyu Xu, Zhuoran Yang, Zhaoran Wang +1
General Value Function (GVF) is a powerful tool to represent both the {\em predictive} and {\em retrospective} knowledge in reinforcement learning (RL). In practice, often multiple…
Provably Faster Algorithms for Bilevel Optimization
Junjie Yang, Kaiyi Ji, Yingbin Liang
Bilevel optimization has been widely applied in many important machine learning applications such as hyperparameter optimization and meta-learning. Recently, several momentum-based…
Proximal Gradient Descent-Ascent: Variable Convergence under KŁ Geometry
Ziyi Chen, Yi Zhou, Tengyu Xu +1
The gradient descent-ascent (GDA) algorithm has been widely applied to solve minimax optimization problems. In order to achieve convergent policy parameters for minimax optimizatio…