89 citations · 302 across the 30 of their papers we have counts for
21 papers · 1 filter
What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity
Kumar Kshitij Patel, Rustem Islamov, Sebastian U Stich +3
Local SGD, also known as Federated Averaging, is a widely used distributed optimization algorithm. Although Local SGD often outperforms alternatives such as Mini-batch SGD in pract…
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
Kumar Kshitij Patel, Margalit Glasgow, Ali Zindari +5
Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD. Despite this success, theoretically…
Pessimistic Value Iteration for Multi-Task Data Sharing in Offline Reinforcement Learning
Chenjia Bai, Lingxiao Wang, Jianye Hao +4
Offline Reinforcement Learning (RL) has shown promising results in learning a task-specific policy from a fixed dataset. However, successful offline RL often relies heavily on the…
Federated Online and Bandit Convex Optimization
Kumar Kshitij Patel, Lingxiao Wang, Aadirupa Saha +1
We study the problems of distributed online and bandit convex optimization against an adaptive adversary. We aim to minimize the average regret on machines working in parallel…
On the Effect of Defections in Federated Learning and How to Prevent Them
Minbiao Han, Kumar Kshitij Patel, Han Shao +1
Federated learning is a machine learning protocol that enables a large population of agents to collaborate over multiple rounds to produce a single consensus model. There are sever…
Bridging the Sim-to-Real Gap from the Information Bottleneck Perspective
Haoran He, Peilin Wu, Chenjia Bai +5
Reinforcement Learning (RL) has recently achieved remarkable success in robotic control. However, most works in RL operate in simulated environments where privileged knowledge (e.g…
An Analysis of Attention via the Lens of Exchangeability and Latent Variable Models
Yufeng Zhang, Boyi Liu, Qi Cai +2
With the attention mechanism, transformers achieve significant empirical successes. Despite the intuitive understanding that transformers perform relational inference over long seq…
Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learning
Shuang Qiu, Lingxiao Wang, Chenjia Bai +2
In view of its power in extracting feature representation, contrastive self-supervised learning has been successfully integrated into the practice of (deep) reinforcement learning…
Embed to Control Partially Observed Systems: Representation Learning with Provable Sample Efficiency
Lingxiao Wang, Qi Cai, Zhuoran Yang +1
Reinforcement learning in partially observed Markov decision processes (POMDPs) faces two challenges. (i) It often takes the full history to predict the future, which induces a sam…
Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learning
Chenjia Bai, Lingxiao Wang, Zhuoran Yang +4
Offline Reinforcement Learning (RL) aims to learn policies from previously collected datasets without exploring the environment. Directly applying off-policy algorithms to offline…
Dynamic Bottleneck for Robust Self-Supervised Exploration
Chenjia Bai, Lingxiao Wang, Lei Han +4
Exploration methods based on pseudo-count of transitions or curiosity of dynamics have achieved promising results in solving reinforcement learning with sparse rewards. However, su…
Adaptive Differentially Private Empirical Risk Minimization
Xiaoxia Wu, Lingxiao Wang, Irina Cristali +2
We propose an adaptive (stochastic) gradient perturbation method for differentially private empirical risk minimization. At each iteration, the random noise added to the gradient i…
False Correlation Reduction for Offline Reinforcement Learning
Zhihong Deng, Zuyue Fu, Lingxiao Wang +5
Offline reinforcement learning (RL) harnesses the power of massive datasets for resolving sequential decision problems. Most existing papers only discuss defending against out-of-d…
Permutation Invariant Policy Optimization for Mean-Field Multi-Agent Reinforcement Learning: A Principled Approach
Yan Li, Lingxiao Wang, Jiachen Yang +4
Multi-agent reinforcement learning (MARL) becomes more challenging in the presence of more agents, as the capacity of the joint state and action spaces grows exponentially in the n…
Principled Exploration via Optimistic Bootstrapping and Backward Induction
Chenjia Bai, Lingxiao Wang, Lei Han +4
One principled approach for provably efficient exploration is incorporating the upper confidence bound (UCB) into the value function as a bonus. However, UCB is specified to deal w…
Variational Dynamic for Self-Supervised Exploration in Deep Reinforcement Learning
Chenjia Bai, Peng Liu, Kaiyu Liu +3
Efficient exploration remains a challenging problem in reinforcement learning, especially for tasks where extrinsic rewards from environments are sparse or even totally disregarded…
On the Global Optimality of Model-Agnostic Meta-Learning
Lingxiao Wang, Qi Cai, Zhuoran Yang +1
Model-agnostic meta-learning (MAML) formulates meta-learning as a bilevel optimization problem, where the inner level solves each subtask based on a shared prior, while the outer l…
Provably Efficient Causal Reinforcement Learning with Confounded Observational Data
Lingxiao Wang, Zhuoran Yang, Zhaoran Wang
Empowered by expressive function approximators such as neural networks, deep reinforcement learning (DRL) achieves tremendous empirical successes. However, learning expressive func…
Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration for Mean-Field Reinforcement Learning
Lingxiao Wang, Zhuoran Yang, Zhaoran Wang
Multi-agent reinforcement learning (MARL) achieves significant empirical successes. However, MARL suffers from the curse of many agents. In this paper, we exploit the symmetry of a…
Efficient Privacy-Preserving Stochastic Nonconvex Optimization
Lingxiao Wang, Bargav Jayaraman, David Evans +1
While many solutions for privacy-preserving convex empirical risk minimization (ERM) have been developed, privacy-preserving nonconvex ERM remains a challenge. We study nonconvex E…
Neural Policy Gradient Methods: Global Optimality and Rates of Convergence
Lingxiao Wang, Qi Cai, Zhuoran Yang +1
Policy gradient methods with actor-critic schemes demonstrate tremendous empirical successes, especially when the actors and critics are parameterized by neural networks. However,…