604 citations · 2k across the 82 of their papers we have counts for
21 papers · 1 filter
Spurious Feature Diversification Improves Out-of-distribution Generalization
Yong Lin, Lu Tan, Yifan Hao +5
Generalization to out-of-distribution (OOD) data is a critical challenge in machine learning. Ensemble-based methods, like weight space ensembles that interpolate model parameters,…
Mitigating the Alignment Tax of RLHF
Yong Lin, Hangyu Lin, Wei Xiong +14
LLMs acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, w…
A Theoretical Analysis of Optimistic Proximal Policy Optimization in Linear Markov Decision Processes
Han Zhong, Tong Zhang
The proximal policy optimization (PPO) algorithm stands as one of the most prosperous methods in the field of reinforcement learning (RL). Despite its success, the theoretical unde…
What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?
Rui Yang, Yong Lin, Xiaoteng Ma +3
Offline goal-conditioned RL (GCRL) offers a way to train general-purpose agents from fully offline datasets. In addition to being conservative within the dataset, the generalizatio…
Effective Bilevel Optimization via Minimax Reformulation
Xiaoyu Wang, Rui Pan, Renjie Pi +1
Bilevel optimization has found successful applications in various machine learning problems, including hyper-parameter optimization, data cleaning, and meta-learning. However, its…
RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
Hanze Dong, Wei Xiong, Deepanshu Goyal +7
Generative foundation models are susceptible to implicit biases that can arise from extensive unsupervised training data. Such biases can produce suboptimal samples, skewed outcome…
Learning in POMDPs is Sample-Efficient with Hindsight Observability
Jonathan N. Lee, Alekh Agarwal, Christoph Dann +1
POMDPs capture a broad class of decision making problems, but hardness results suggest that learning is intractable even in simple settings due to the inherent partial observabilit…
Probabilistic Bilevel Coreset Selection
Xiao Zhou, Renjie Pi, Weizhong Zhang +2
The goal of coreset selection in supervised learning is to produce a weighted subset of data, so that training only on the subset achieves similar performance as training on the en…
Normalizing Flow with Variational Latent Representation
Hanze Dong, Shizhe Diao, Weizhong Zhang +1
Normalizing flow (NF) has gained popularity over traditional maximum likelihood based methods due to its strong capability to model complex data distributions. However, the standar…
GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond
Han Zhong, Wei Xiong, Sirui Zheng +4
We study sample efficient reinforcement learning (RL) under the general framework of interactive decision making, which includes Markov decision process (MDP), partially observable…
A Self-Play Posterior Sampling Algorithm for Zero-Sum Markov Games
Wei Xiong, Han Zhong, Chengshuai Shi +2
Existing studies on provably efficient algorithms for Markov games (MGs) almost exclusively build on the "optimism in the face of uncertainty" (OFU) principle. This work focuses on…
A Provably Efficient Model-Free Posterior Sampling Method for Episodic Reinforcement Learning
Christoph Dann, Mehryar Mohri, Tong Zhang +1
Thompson Sampling is one of the most effective methods for contextual bandits and has been generalized to posterior sampling for certain MDP settings. However, existing posterior s…
Model-based RL with Optimistic Posterior Sampling: Structural Conditions and Sample Complexity
Alekh Agarwal, Tong Zhang
We propose a general framework to design posterior sampling methods for model-based RL. We show that the proposed algorithms can be analyzed by reducing regret to Hellinger distanc…
Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game
Wei Xiong, Han Zhong, Chengshuai Shi +3
Offline reinforcement learning (RL) aims at learning an optimal strategy using a pre-collected dataset without further interactions with the environment. While various algorithms h…
Non-Linear Reinforcement Learning in Large Action Spaces: Structural Conditions and Sample-efficiency of Posterior Sampling
Alekh Agarwal, Tong Zhang
Provably sample-efficient Reinforcement Learning (RL) with rich observations and function approximation has witnessed tremendous recent progress, particularly when the underlying f…
MAP Inference via L2-Sphere Linear Program Reformulation
Baoyuan Wu, Li Shen, Tong Zhang +1
Maximum a posteriori (MAP) inference is an important task for graphical models. Due to complex dependencies among variables in realistic model, finding an exact solution for MAP in…
Hessian-Aware Zeroth-Order Optimization for Black-Box Adversarial Attack
Haishan Ye, Zhichao Huang, Cong Fang +2
Zeroth-order optimization is an important research topic in machine learning. In recent years, it has become a key tool in black-box adversarial attack to neural network based imag…
Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space
Jiechao Xiong, Qing Wang, Zhuoran Yang +7
Most existing deep reinforcement learning (DRL) frameworks consider either discrete action space or continuous action space solely. Motivated by applications in computer games, we…
Fully Implicit Online Learning
Chaobing Song, Ji Liu, Han Liu +2
Regularized online learning is widely used in machine learning applications. In online learning, performing exact minimization ( implicit update) is known to be beneficial t…
When Work Matters: Transforming Classical Network Structures to Graph CNN
Wenting Zhao, Chunyan Xu, Zhen Cui +4
Numerous pattern recognition applications can be formed as learning from graph-structured data, including social network, protein-interaction network, the world wide web data, know…
Adjusting Leverage Scores by Row Weighting: A Practical Approach to Coherent Matrix Completion
Shusen Wang, Tong Zhang, Zhihua Zhang
Low-rank matrix completion is an important problem with extensive real-world applications. When observations are uniformly sampled from the underlying matrix entries, existing meth…