89 citations · 216 across the 11 of their papers we have counts for
11 papers · 1 filter
An Analysis of Attention via the Lens of Exchangeability and Latent Variable Models
Yufeng Zhang, Boyi Liu, Qi Cai +2
With the attention mechanism, transformers achieve significant empirical successes. Despite the intuitive understanding that transformers perform relational inference over long seq…
Embed to Control Partially Observed Systems: Representation Learning with Provable Sample Efficiency
Lingxiao Wang, Qi Cai, Zhuoran Yang +1
Reinforcement learning in partially observed Markov decision processes (POMDPs) faces two challenges. (i) It often takes the full history to predict the future, which induces a sam…
Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiency
Qi Cai, Zhuoran Yang, Zhaoran Wang
We study reinforcement learning for partially observed Markov decision processes (POMDPs) with infinite observation and state spaces, which remains less investigated theoretically.…
On the Global Optimality of Model-Agnostic Meta-Learning
Lingxiao Wang, Qi Cai, Zhuoran Yang +1
Model-agnostic meta-learning (MAML) formulates meta-learning as a bilevel optimization problem, where the inner level solves each subtask based on a shared prior, while the outer l…
Can Temporal-Difference and Q-Learning Learn Representation? A Mean-Field Theory
Yufeng Zhang, Qi Cai, Zhuoran Yang +2
Temporal-difference and Q-learning play a key role in deep reinforcement learning, where they are empowered by expressive nonlinear function approximators such as neural networks.…
Generative Adversarial Imitation Learning with Neural Networks: Global Optimality and Convergence Rate
Yufeng Zhang, Qi Cai, Zhuoran Yang +1
Generative adversarial imitation learning (GAIL) demonstrates tremendous success in practice, especially when combined with neural networks. Different from reinforcement learning,…
Provably Efficient Exploration in Policy Optimization
Qi Cai, Zhuoran Yang, Chi Jin +1
While policy-based reinforcement learning (RL) achieves tremendous successes in practice, it is significantly less understood in theory, especially compared with value-based RL. In…
Neural Policy Gradient Methods: Global Optimality and Rates of Convergence
Lingxiao Wang, Qi Cai, Zhuoran Yang +1
Policy gradient methods with actor-critic schemes demonstrate tremendous empirical successes, especially when the actors and critics are parameterized by neural networks. However,…
Neural Proximal/Trust Region Policy Optimization Attains Globally Optimal Policy
Boyi Liu, Qi Cai, Zhuoran Yang +1
Proximal policy optimization and trust region policy optimization (PPO and TRPO) with actor and critic parametrized by neural networks achieve significant empirical success in deep…
Neural Temporal-Difference and Q-Learning Provably Converge to Global Optima
Qi Cai, Zhuoran Yang, Jason D. Lee +1
Temporal-difference learning (TD), coupled with neural networks, is among the most fundamental building blocks of deep reinforcement learning. However, due to the nonlinearity in v…
On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator
Qi Cai, Mingyi Hong, Yongxin Chen +1
We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges aris…