1 citations · 1 across the 18 of their papers we have counts for
21 papers
Is Separation Necessary for Gradient EM to Learn Gaussian Mixtures in High Dimensions?
Yiran Zhang, Mo Zhou, Weihang Xu +2
Learning Gaussian mixture models (GMMs) using the Expectation-Maximization (EM) algorithm and its gradient-based variants is a fundamental problem in machine learning. It is known…
Asymptotically Optimal Regret for Reinforcement Learning without Horizon Dependence
Runlong Zhou, Zihan Zhang, Maryam Fazel +1
We study horizon-free regret minimization for finite-horizon time-homogeneous tabular Markov decision processes with states, actions, horizon , and per-trajectory total…
Cold-Start Personalization via Training-Free Priors from Structured World Models
Avinandan Bose, Shuyue Stella Li, Faeze Brahman +6
Cold-start personalization requires inferring user preferences through interaction when no user-specific historical data is available. The core challenge is a routing problem: each…
Convergence Dynamics of Over-Parameterized Score Matching for a Single Gaussian
Yiran Zhang, Weihang Xu, Mo Zhou +2
Score matching has become a central training objective in modern generative modeling, particularly in diffusion models, where it is used to learn high-dimensional data distribution…
Global Convergence of Four-Layer Matrix Factorization under Random Initialization
Minrui Luo, Weihang Xu, Xiang Gao +2
Gradient descent dynamics on the deep matrix factorization problem is extensively studied as a simplified theoretical model for deep neural networks. Although the convergence theor…
Policy-Based Trajectory Clustering in Offline Reinforcement Learning
Hao Hu, Xinqi Wang, Simon Shaolei Du
We introduce a novel task of clustering trajectories from offline reinforcement learning (RL) datasets, where each cluster center represents the policy that generated its trajector…
Sharp Gap-Dependent Variance-Aware Regret Bounds for Tabular MDPs
Shulun Chen, Runlong Zhou, Zihan Zhang +2
We consider the gap-dependent regret bounds for episodic MDPs. We show that the Monotonic Value Propagation (MVP) algorithm achieves a variance-aware gap-dependent regret bound of…
Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixtures
Mo Zhou, Weihang Xu, Maryam Fazel +1
Learning Gaussian Mixture Models (GMMs) is a fundamental problem in statistics and machine learning, with the Expectation-Maximization (EM) algorithm and its popular variant gradie…
LoRe: Personalizing LLMs via Low-Rank Reward Modeling
Avinandan Bose, Zhihan Xiong, Yuejie Chi +3
Personalizing large language models (LLMs) to accommodate diverse user preferences is essential for enhancing alignment and user satisfaction. Traditional reinforcement learning fr…
A Minimalist Example of Edge-of-Stability and Progressive Sharpening
Liming Liu, Zixuan Zhang, Simon Du +1
Recent advances in deep learning optimization have unveiled two intriguing phenomena under large learning rates: Edge of Stability (EoS) and Progressive Sharpening (PS), challengin…
Extragradient Preference Optimization (EGPO): Beyond Last-Iterate Convergence for Nash Learning from Human Feedback
Runlong Zhou, Maryam Fazel, Simon S. Du
Reinforcement learning from human feedback (RLHF) has become essential for improving language model capabilities, but traditional approaches rely on the assumption that human prefe…
Towards Understanding the Benefit of Multitask Representation Learning in Decision Process
Rui Lu, Yang Yue, Andrew Zhao +2
Multitask Representation Learning (MRL) has emerged as a prevalent technique to improve sample efficiency in Reinforcement Learning (RL). Empirical studies have found that training…
Hybrid Preference Optimization for Alignment: Provably Faster Convergence Rates by Combining Offline Preferences with Online Exploration
Avinandan Bose, Zhihan Xiong, Aadirupa Saha +2
Reinforcement Learning from Human Feedback (RLHF) is currently the leading approach for aligning large language models with human preferences. Typically, these models rely on exten…
Learning to Cooperate with Humans using Generative Agents
Yancheng Liang, Daphne Chen, Abhishek Gupta +2
Training agents that can coordinate zero-shot with humans is a key mission in multi-agent reinforcement learning (MARL). Current algorithms focus on training simulated human partne…
Preference-Based Multi-Agent Reinforcement Learning: Data Coverage and Algorithmic Techniques
Natalia Zhang, Xinqi Wang, Qiwen Cui +3
We initiate the study of Preference-Based Multi-Agent Reinforcement Learning (PbMARL), exploring both theoretical foundations and empirical validations. We define the task as ident…
Toward Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixture Models
Weihang Xu, Maryam Fazel, Simon S. Du
We study the gradient Expectation-Maximization (EM) algorithm for Gaussian Mixture Models (GMM) in the over-parameterized setting, where a general GMM with components learns…
Distributional Successor Features Enable Zero-Shot Policy Optimization
Chuning Zhu, Xinqi Wang, Tyler Han +2
Intelligent agents must be generalists, capable of quickly adapting to various tasks. In reinforcement learning (RL), model-based RL learns a dynamics model of the world, in princi…
Offline Multi-task Transfer RL with Representational Penalization
Avinandan Bose, Simon Shaolei Du, Maryam Fazel
We study the problem of representation transfer in offline Reinforcement Learning (RL), where a learner has access to episodic data from a number of source tasks collected a priori…
Learning Optimal Tax Design in Nonatomic Congestion Games
Qiwen Cui, Maryam Fazel, Simon S. Du
In multiplayer games, self-interested behavior among the players can harm the social welfare. Tax mechanisms are a common method to alleviate this issue and induce socially optimal…
Refined Sample Complexity for Markov Games with Independent Linear Function Approximation
Yan Dai, Qiwen Cui, Simon S. Du
Markov Games (MG) is an important model for Multi-Agent Reinforcement Learning (MARL). It was long believed that the "curse of multi-agents" (i.e., the algorithmic performance drop…
Free from Bellman Completeness: Trajectory Stitching via Model-based Return-conditioned Supervised Learning
Zhaoyi Zhou, Chuning Zhu, Runlong Zhou +3
Off-policy dynamic programming (DP) techniques such as -learning have proven to be important in sequential decision-making problems. In the presence of function approximation, h…