124 citations · 953 across the 47 of their papers we have counts for
47 papers
Sparse PCA with Oracle Property
Quanquan Gu, Zhaoran Wang, Han Liu
In this paper, we study the estimation of the -dimensional sparse principal subspace of covariance matrix in the high-dimensional setting. We aim to recover the oracle princ…
Doubly Robust Off-Policy Actor-Critic: Convergence and Optimality
Tengyu Xu, Zhuoran Yang, Zhaoran Wang +1
Designing off-policy reinforcement learning algorithms is typically a very challenging task, because a desirable iteration update often involves an expectation over an on-policy di…
Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning
Luofeng Liao, Zuyue Fu, Zhuoran Yang +3
In offline reinforcement learning (RL) an optimal policy is learned solely from a priori collected observational data. However, in observational data, actions are often confounded…
Provably Training Overparameterized Neural Network Classifiers with Non-convex Constraints
You-Lin Chen, Zhaoran Wang, Mladen Kolar
Training a classifier under non-convex constraints has gotten increasing attention in the machine learning community thanks to its wide range of applications such as algorithmic fa…
Risk-Sensitive Deep RL: Variance-Constrained Actor-Critic Provably Finds Globally Optimal Policy
Han Zhong, Xun Deng, Ethan X. Fang +3
While deep reinforcement learning has achieved tremendous successes in various applications, most existing works only focus on maximizing the expected value of total return and thu…
Variational Transport: A Convergent Particle-BasedAlgorithm for Distributional Optimization
Zhuoran Yang, Yufeng Zhang, Yongxin Chen +1
We consider the optimization problem of minimizing a functional defined over a family of probability distributions, where the objective functional is assumed to possess a variation…
Provable Fictitious Play for General Mean-Field Games
Qiaomin Xie, Zhuoran Yang, Zhaoran Wang +1
We propose a reinforcement learning algorithm for stationary mean-field games, where the goal is to learn a pair of mean-field state and stationary policy that constitutes the Nash…
Global Convergence of Policy Gradient for Linear-Quadratic Mean-Field Control/Game in Continuous Time
Weichen Wang, Jiequn Han, Zhuoran Yang +1
Reinforcement learning is a powerful tool to learn the optimal policy of possibly multiple agents by interacting with the environment. As the number of agents grow to be very large…
Accelerating Nonconvex Learning via Replica Exchange Langevin Diffusion
Yi Chen, Jinglin Chen, Jing Dong +2
Langevin diffusion is a powerful method for nonconvex optimization, which enables the escape from local minima by injecting noise into the gradient. In particular, the temperature…
A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic
Mingyi Hong, Hoi-To Wai, Zhaoran Wang +1
This paper analyzes a two-timescale stochastic algorithm framework for bilevel optimization. Bilevel optimization is a class of problems which exhibit a two-level structure, and it…
Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach
Luofeng Liao, You-Lin Chen, Zhuoran Yang +3
Structural equation models (SEMs) are widely used in sciences, ranging from economics to psychology, to uncover causal relationships underlying a complex system under consideration…
On the Global Optimality of Model-Agnostic Meta-Learning
Lingxiao Wang, Qi Cai, Zhuoran Yang +1
Model-agnostic meta-learning (MAML) formulates meta-learning as a bilevel optimization problem, where the inner level solves each subtask based on a shared prior, while the outer l…
Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regret
Yingjie Fei, Zhuoran Yang, Yudong Chen +2
We study risk-sensitive reinforcement learning in episodic Markov decision processes with unknown transition kernels, where the goal is to optimize the total reward under the risk…
Provably Efficient Causal Reinforcement Learning with Confounded Observational Data
Lingxiao Wang, Zhuoran Yang, Zhaoran Wang
Empowered by expressive function approximators such as neural networks, deep reinforcement learning (DRL) achieves tremendous empirical successes. However, learning expressive func…
Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration for Mean-Field Reinforcement Learning
Lingxiao Wang, Zhuoran Yang, Zhaoran Wang
Multi-agent reinforcement learning (MARL) achieves significant empirical successes. However, MARL suffers from the curse of many agents. In this paper, we exploit the symmetry of a…
Can Temporal-Difference and Q-Learning Learn Representation? A Mean-Field Theory
Yufeng Zhang, Qi Cai, Zhuoran Yang +2
Temporal-difference and Q-learning play a key role in deep reinforcement learning, where they are empowered by expressive nonlinear function approximators such as neural networks.…
Upper Confidence Primal-Dual Reinforcement Learning for CMDP with Adversarial Loss
Shuang Qiu, Xiaohan Wei, Zhuoran Yang +2
We consider online learning for episodic stochastically constrained Markov decision processes (CMDPs), which plays a central role in ensuring the safety of reinforcement learning.…
Provably Efficient Safe Exploration via Primal-Dual Policy Optimization
Dongsheng Ding, Xiaohan Wei, Zhuoran Yang +2
We study the Safe Reinforcement Learning (SRL) problem using the Constrained Markov Decision Process (CMDP) formulation in which an agent aims to maximize the expected total reward…
Actor-Critic Provably Finds Nash Equilibria of Linear-Quadratic Mean-Field Games
Zuyue Fu, Zhuoran Yang, Yongxin Chen +1
We study discrete-time mean-field Markov games with infinite numbers of agents where each agent aims to minimize its ergodic cost. We consider the setting where the agents have ide…
Sample Elicitation
Jiaheng Wei, Zuyue Fu, Yang Liu +3
It is important to collect credible training samples for building data-intensive learning systems (e.g., a deep learning system). Asking people to report complex distributi…
Neural Policy Gradient Methods: Global Optimality and Rates of Convergence
Lingxiao Wang, Qi Cai, Zhuoran Yang +1
Policy gradient methods with actor-critic schemes demonstrate tremendous empirical successes, especially when the actors and critics are parameterized by neural networks. However,…
Fast Multi-Agent Temporal-Difference Learning via Homotopy Stochastic Primal-Dual Optimization
Dongsheng Ding, Xiaohan Wei, Zhuoran Yang +2
We study the policy evaluation problem in multi-agent reinforcement learning where a group of agents, with jointly observed states and private local actions and rewards, collaborat…
More Supervision, Less Computation: Statistical-Computational Tradeoffs in Weakly Supervised Learning
Xinyang Yi, Zhaoran Wang, Zhuoran Yang +2
We consider the weakly supervised binary classification problem where the labels are randomly flipped with probability . Although there exist numerous algorithms for this pro…
On the Global Convergence of Actor-Critic: A Case for Linear Quadratic Regulator with Ergodic Cost
Zhuoran Yang, Yongxin Chen, Mingyi Hong +1
Despite the empirical success of the actor-critic algorithm, its theoretical understanding lags behind. In a broader context, actor-critic can be viewed as an online alternating up…