26 citations · 110 across the 26 of their papers we have counts for
27 papers
dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learning
Shirui Chen, Jiantao Jiao, Lillian J. Ratliff +1
Masked diffusion language models (MDLMs) offer the potential for parallel token generation, but most open-source MDLMs decode fewer than 5 tokens per model forward pass even with s…
Efficient Prompt Caching via Embedding Similarity
Hanlin Zhu, Banghua Zhu, Jiantao Jiao
Large language models (LLMs) have achieved huge success in numerous natural language process (NLP) tasks. However, it faces the challenge of significant resource consumption during…
Generative AI Security: Challenges and Countermeasures
Banghua Zhu, Norman Mu, Jiantao Jiao +1
Generative AI's expanding footprint across numerous industries has led to both excitement and increased scrutiny. This paper delves into the unique security challenges posed by Gen…
Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF
Banghua Zhu, Michael I. Jordan, Jiantao Jiao
Reinforcement Learning from Human Feedback (RLHF) is a pivotal technique that aligns language models closely with human-centric values. The initial phase of RLHF involves learning…
Towards Optimal Statistical Watermarking
Baihe Huang, Hanlin Zhu, Banghua Zhu +4
We study statistical watermarking by formulating it as a hypothesis testing problem, a general framework which subsumes all previous statistical watermarking methods. Key to our fo…
Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration
Jinning Li, Xinyi Liu, Banghua Zhu +4
Safe Reinforcement Learning (RL) aims to find a policy that achieves high rewards while satisfying cost constraints. When learning from scratch, safe RL agents tend to be overly co…
Pairwise Proximal Policy Optimization: Harnessing Relative Feedback for LLM Alignment
Tianhao Wu, Banghua Zhu, Ruoyu Zhang +3
Large Language Models (LLMs) can acquire extensive world knowledge through pre-training on large corpora. However, due to exposure to low-quality data, LLMs may exhibit harmful beh…
Noisy Computing of the and Functions
Banghua Zhu, Ziao Wang, Nadim Ghaddar +2
We consider the problem of computing a function of variables using noisy queries, where each query is incorrect with some fixed and known probability . Specifica…
On the Optimal Bounds for Noisy Computing
Banghua Zhu, Ziao Wang, Nadim Ghaddar +2
We revisit the problem of computing with noisy information considered in Feige et al. 1994, which includes computing the OR function from noisy queries, and computing the MAX, SEAR…
Fine-Tuning Language Models with Advantage-Induced Policy Alignment
Banghua Zhu, Hiteshi Sharma, Felipe Vieira Frujeri +4
Reinforcement learning from human feedback (RLHF) has emerged as a reliable approach to aligning large language models (LLMs) to human preferences. Among the plethora of RLHF techn…
On Optimal Caching and Model Multiplexing for Large Model Inference
Banghua Zhu, Ying Sheng, Lianmin Zheng +3
Large Language Models (LLMs) and other large foundation models have achieved noteworthy success, but their size exacerbates existing resource consumption and latency challenges. In…
Doubly Robust Self-Training
Banghua Zhu, Mingyu Ding, Philip Jacobson +4
Self-training is an important technique for solving semi-supervised learning problems. It leverages unlabeled data by generating pseudo-labels and combining them with a limited lab…
Online Learning in a Creator Economy
Banghua Zhu, Sai Praneeth Karimireddy, Jiantao Jiao +1
The creator economy has revolutionized the way individuals can profit through online platforms. In this paper, we initiate the study of online learning in the creator economy by mo…
Online Learning in Stackelberg Games with an Omniscient Follower
Geng Zhao, Banghua Zhu, Jiantao Jiao +1
We study the problem of online learning in a two-player decentralized cooperative Stackelberg game. In each round, the leader first takes an action, followed by the follower who ta…
Principled Reinforcement Learning with Human Feedback from Pairwise or -wise Comparisons
Banghua Zhu, Jiantao Jiao, Michael I. Jordan
We provide a theoretical framework for Reinforcement Learning with Human Feedback (RLHF). Our analysis shows that when the true reward function is linear, the widely used maximum l…
The Sample Complexity of Online Contract Design
Banghua Zhu, Stephen Bates, Zhuoran Yang +3
We study the hidden-action principal-agent problem in an online setting. In each round, the principal posts a contract that specifies the payment to the agent based on each outcome…
Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guarantees
Banghua Zhu, Lun Wang, Qi Pang +4
We propose Byzantine-robust federated learning protocols with nearly optimal statistical rates. In contrast to prior work, our proposed protocols improve the dimension dependence a…
Jump-Start Reinforcement Learning
Ikechukwu Uchendu, Ted Xiao, Yao Lu +9
Reinforcement learning (RL) provides a theoretical framework for continuously improving an agent's behavior via trial and error. However, efficiently learning policies from scratch…
Robust Estimation for Nonparametric Families via Generative Adversarial Networks
Banghua Zhu, Jiantao Jiao, Michael I. Jordan
We provide a general framework for designing Generative Adversarial Networks (GANs) to solve high dimensional robust statistics problems, which aim at estimating unknown parameter…
Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism
Paria Rashidinejad, Banghua Zhu, Cong Ma +2
Offline (or batch) reinforcement learning (RL) algorithms seek to learn an optimal policy from a fixed dataset without active data collection. Based on the composition of the offli…
Minimax Off-Policy Evaluation for Multi-Armed Bandits
Cong Ma, Banghua Zhu, Jiantao Jiao +1
We study the problem of off-policy evaluation in the multi-armed bandit model with bounded rewards, and develop minimax rate-optimal procedures under three settings. First, when th…
Linear Representation Meta-Reinforcement Learning for Instant Adaptation
Matt Peng, Banghua Zhu, Jiantao Jiao
This paper introduces Fast Linearized Adaptive Policy (FLAP), a new meta-reinforcement learning (meta-RL) method that is able to extrapolate well to out-of-distribution tasks witho…
Robust estimation via generalized quasi-gradients
Banghua Zhu, Jiantao Jiao, Jacob Steinhardt
We explore why many recently proposed robust estimation problems are efficiently solvable, even though the underlying optimization problems are non-convex. We study the loss landsc…
When does the Tukey median work?
Banghua Zhu, Jiantao Jiao, Jacob Steinhardt
We analyze the performance of the Tukey median estimator under total variation (TV) distance corruptions. Previous results show that under Huber's additive corruption model, the br…