921 citations · 1.7k across the 59 of their papers we have counts for
62 papers
Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration
Jinning Li, Xinyi Liu, Banghua Zhu +4
Safe Reinforcement Learning (RL) aims to find a policy that achieves high rewards while satisfying cost constraints. When learning from scratch, safe RL agents tend to be overly co…
Pairwise Proximal Policy Optimization: Harnessing Relative Feedback for LLM Alignment
Tianhao Wu, Banghua Zhu, Ruoyu Zhang +3
Large Language Models (LLMs) can acquire extensive world knowledge through pre-training on large corpora. However, due to exposure to low-quality data, LLMs may exhibit harmful beh…
Noisy Computing of the and Functions
Banghua Zhu, Ziao Wang, Nadim Ghaddar +2
We consider the problem of computing a function of variables using noisy queries, where each query is incorrect with some fixed and known probability . Specifica…
On the Optimal Bounds for Noisy Computing
Banghua Zhu, Ziao Wang, Nadim Ghaddar +2
We revisit the problem of computing with noisy information considered in Feige et al. 1994, which includes computing the OR function from noisy queries, and computing the MAX, SEAR…
Fine-Tuning Language Models with Advantage-Induced Policy Alignment
Banghua Zhu, Hiteshi Sharma, Felipe Vieira Frujeri +4
Reinforcement learning from human feedback (RLHF) has emerged as a reliable approach to aligning large language models (LLMs) to human preferences. Among the plethora of RLHF techn…
On Optimal Caching and Model Multiplexing for Large Model Inference
Banghua Zhu, Ying Sheng, Lianmin Zheng +3
Large Language Models (LLMs) and other large foundation models have achieved noteworthy success, but their size exacerbates existing resource consumption and latency challenges. In…
Doubly Robust Self-Training
Banghua Zhu, Mingyu Ding, Philip Jacobson +4
Self-training is an important technique for solving semi-supervised learning problems. It leverages unlabeled data by generating pseudo-labels and combining them with a limited lab…
Online Learning in a Creator Economy
Banghua Zhu, Sai Praneeth Karimireddy, Jiantao Jiao +1
The creator economy has revolutionized the way individuals can profit through online platforms. In this paper, we initiate the study of online learning in the creator economy by mo…
Statistical Complexity and Optimal Algorithms for Non-linear Ridge Bandits
Nived Rajaraman, Yanjun Han, Jiantao Jiao +1
We consider the sequential decision-making problem where the mean outcome is a non-linear function of the chosen action. Compared with the linear model, two curious phenomena arise…
Importance Weighted Actor-Critic for Optimal Conservative Offline Reinforcement Learning
Hanlin Zhu, Paria Rashidinejad, Jiantao Jiao
We propose A-Crab (Actor-Critic Regularized by Average Bellman error), a new practical algorithm for offline reinforcement learning (RL) in complex environments with insufficient d…
Online Learning in Stackelberg Games with an Omniscient Follower
Geng Zhao, Banghua Zhu, Jiantao Jiao +1
We study the problem of online learning in a two-player decentralized cooperative Stackelberg game. In each round, the leader first takes an action, followed by the follower who ta…
Principled Reinforcement Learning with Human Feedback from Pairwise or -wise Comparisons
Banghua Zhu, Jiantao Jiao, Michael I. Jordan
We provide a theoretical framework for Reinforcement Learning with Human Feedback (RLHF). Our analysis shows that when the true reward function is linear, the widely used maximum l…
The Sample Complexity of Online Contract Design
Banghua Zhu, Stephen Bates, Zhuoran Yang +3
We study the hidden-action principal-agent problem in an online setting. In each round, the principal posts a contract that specifies the payment to the agent based on each outcome…
Beyond the Best: Estimating Distribution Functionals in Infinite-Armed Bandits
Yifei Wang, Tavor Baharav, Yanjun Han +2
In the infinite-armed bandit problem, each arm's average reward is sampled from an unknown distribution, and each arm can be sampled further to obtain noisy estimates of the averag…
Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangian
Paria Rashidinejad, Hanlin Zhu, Kunhe Yang +2
Offline reinforcement learning (RL), which refers to decision-making from a previously-collected dataset of interactions, has received significant attention over the past years. Mu…
Minimax Optimal Online Imitation Learning via Replay Estimation
Gokul Swamy, Nived Rajaraman, Matthew Peng +5
Online imitation learning is the problem of how best to mimic expert demonstrations, given access to the environment or an accurate simulator. Prior work has shown that in the infi…
Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guarantees
Banghua Zhu, Lun Wang, Qi Pang +4
We propose Byzantine-robust federated learning protocols with nearly optimal statistical rates. In contrast to prior work, our proposed protocols improve the dimension dependence a…
Jump-Start Reinforcement Learning
Ikechukwu Uchendu, Ted Xiao, Yao Lu +9
Reinforcement learning (RL) provides a theoretical framework for continuously improving an agent's behavior via trial and error. However, efficiently learning policies from scratch…
Robust Estimation for Nonparametric Families via Generative Adversarial Networks
Banghua Zhu, Jiantao Jiao, Michael I. Jordan
We provide a general framework for designing Generative Adversarial Networks (GANs) to solve high dimensional robust statistics problems, which aim at estimating unknown parameter…
Nearly Optimal Policy Optimization with Stable at Any Time Guarantee
Tianhao Wu, Yunchang Yang, Han Zhong +3
Policy optimization methods are one of the most widely used classes of Reinforcement Learning (RL) algorithms. However, theoretical understanding of these methods remains insuffici…
Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learning
Yuexiang Zhai, Christina Baek, Zhengyuan Zhou +2
Many goal-reaching reinforcement learning (RL) tasks have empirically verified that rewarding the agent on subgoals improves convergence speed and practical performance. We attempt…
MADE: Exploration via Maximizing Deviation from Explored Regions
Tianjun Zhang, Paria Rashidinejad, Jiantao Jiao +3
In online reinforcement learning (RL), efficient exploration remains particularly challenging in high-dimensional environments with sparse rewards. In low-dimensional environments,…
Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learning
Jinhyun So, Ramy E. Ali, Basak Guler +2
Secure aggregation is a critical component in federated learning (FL), which enables the server to learn the aggregate model of the users without observing their local models. Conv…
Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism
Paria Rashidinejad, Banghua Zhu, Cong Ma +2
Offline (or batch) reinforcement learning (RL) algorithms seek to learn an optimal policy from a fixed dataset without active data collection. Based on the composition of the offli…