378 citations · 2.2k across the 69 of their papers we have counts for
27 papers · 1 filter
Offline Reinforcement Learning with On-Policy Q-Function Regularization
Laixi Shi, Robert Dadashi, Yuejie Chi +2
The core challenge of offline reinforcement learning (RL) is dealing with the (potentially catastrophic) extrapolation error induced by the distribution shift between the history d…
Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlation
Wenhao Ding, Laixi Shi, Yuejie Chi +1
Robustness has been extensively studied in reinforcement learning (RL) to handle various forms of uncertainty such as random perturbations, rare events, and malicious attacks. In t…
Understanding Masked Autoencoders via Hierarchical Latent Variable Models
Lingjing Kong, Martin Q. Ma, Guangyi Chen +4
Masked autoencoder (MAE), a simple and effective self-supervised learning framework based on the reconstruction of masked image regions, has recently achieved prominent success in…
Identification of Nonlinear Latent Hierarchical Models
Lingjing Kong, Biwei Huang, Feng Xie +3
Identifying latent variables and causal structures from observational data is essential to many real-world applications involving biological data, medical data, and unstructured da…
Reward-agnostic Fine-tuning: Provable Statistical Benefits of Hybrid Reinforcement Learning
Gen Li, Wenhao Zhan, Jason D. Lee +2
This paper studies tabular reinforcement learning (RL) in the hybrid setting, which assumes access to both an offline dataset and online interactions with the unknown environment.…
Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication Compression
Boyue Li, Yuejie Chi
Achieving communication efficiency in decentralized machine learning has been attracting significant attention, with communication compression recognized as an effective technique…
The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Model
Laixi Shi, Gen Li, Yuting Wei +3
This paper investigates model robustness in reinforcement learning (RL) to reduce the sim-to-real gap in practice. We adopt the framework of distributionally robust Markov decision…
The Blessing of Heterogeneity in Federated Q-Learning: Linear Speedup and Beyond
Jiin Woo, Gauri Joshi, Yuejie Chi
When the data used for reinforcement learning (RL) are collected by multiple agents in a distributed manner, federated versions of RL algorithms allow collaborative learning withou…
The Power of Preconditioning in Overparameterized Low-Rank Matrix Sensing
Xingyu Xu, Yandi Shen, Yuejie Chi +1
We propose $\textsf{ScaledGD($λ$)}$, a preconditioned gradient descent method to tackle the low-rank matrix sensing problem when the true rank is unknown, and when the matrix is po…
Fast Computation of Optimal Transport via Entropy-Regularized Extragradient Methods
Gen Li, Yanxi Chen, Yu Huang +3
Efficient computation of the optimal transport distance between two distributions serves as an algorithm subroutine that empowers various applications. This paper develops a scalab…
Minimax-Optimal Multi-Agent RL in Markov Games With a Generative Model
Gen Li, Yuejie Chi, Yuting Wei +1
This paper studies multi-agent reinforcement learning in Markov games, with the goal of learning Nash equilibria or coarse correlated equilibria (CCE) sample-optimally. All prior r…
Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexity
Laixi Shi, Yuejie Chi
This paper concerns the central issues of model robustness and sample efficiency in offline reinforcement learning (RL), which aims to learn to perform decision making from history…
SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression
Zhize Li, Haoyu Zhao, Boyue Li +1
To enable large-scale machine learning in bandwidth-hungry environments such as wireless networks, significant progress has been made recently in designing communication-efficient…
Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexity
Laixi Shi, Gen Li, Yuting Wei +2
Offline or batch reinforcement learning seeks to learn a near-optimal policy using history data without active exploration of the environment. To counter the insufficient coverage…
BEER: Fast Rate for Decentralized Nonconvex Optimization with Communication Compression
Haoyu Zhao, Boyue Li, Zhize Li +2
Communication efficiency has been widely recognized as the bottleneck for large-scale decentralized machine learning applications in multi-agent or federated environments. To tackl…
Breaking the Sample Complexity Barrier to Regret-Optimal Model-Free Reinforcement Learning
Gen Li, Laixi Shi, Yuxin Chen +1
Achieving sample efficiency in online episodic reinforcement learning (RL) requires optimally balancing exploration and exploitation. When it comes to a finite-horizon episodic Mar…
Policy Mirror Descent for Regularized Reinforcement Learning: A Generalized Framework with Linear Convergence
Wenhao Zhan, Shicong Cen, Baihe Huang +3
Policy optimization, which finds the desired policy by maximizing value functions via optimization techniques, lies at the heart of reinforcement learning (RL). In addition to valu…
Sample-Efficient Reinforcement Learning Is Feasible for Linearly Realizable MDPs with Limited Revisiting
Gen Li, Yuxin Chen, Yuejie Chi +2
Low-complexity models such as linear function representation play a pivotal role in enabling sample-efficient reinforcement learning (RL). The current paper pertains to a scenario…
Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurements
Tian Tong, Cong Ma, Ashley Prater-Bennette +2
Tensors, which provide a powerful and flexible model for representing multi-attribute data and multi-way interactions, play an indispensable role in modern data science across vari…
Softmax Policy Gradient Methods Can Take Exponential Time to Converge
Gen Li, Yuting Wei, Yuejie Chi +1
The softmax policy gradient (PG) method, which performs gradient ascent under softmax policy parameterization, is arguably one of the de facto implementations of policy optimizatio…
Low-Rank Matrix Recovery with Scaled Subgradient Methods: Fast and Robust Convergence Without the Condition Number
Tian Tong, Cong Ma, Yuejie Chi
Many problems in data science can be treated as estimating a low-rank matrix from highly incomplete, sometimes even corrupted, observations. One popular approach is to resort to ma…
Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reduction
Gen Li, Yuting Wei, Yuejie Chi +2
Asynchronous Q-learning aims to learn the optimal action-value function (or Q-function) of a Markov decision process (MDP), based on a single trajectory of Markovian samples induce…
Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model
Gen Li, Yuting Wei, Yuejie Chi +1
This paper is concerned with the sample efficiency of reinforcement learning, assuming access to a generative model (or simulator). We first consider -discounted infinite-horizo…
Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent
Tian Tong, Cong Ma, Yuejie Chi
Low-rank matrix estimation is a canonical problem that finds numerous applications in signal processing, machine learning and imaging science. A popular approach in practice is to…