6 citations · 11 across the 22 of their papers we have counts for
26 papers
Relative Generalization Invariance of LLM Pretraining
Fengzhuo Zhang, Shuche Wang, Shenggui Li +7
Large Language Model (LLM) pretraining performance is jointly shaped by three components of the training triplet: the optimizer, model architecture, and training data stream. Howev…
Rethinking the Trust Region in LLM Reinforcement Learning
Penghui Qi, Xiangxin Zhou, Zichen Liu +4
Reinforcement learning (RL) has become a cornerstone for fine-tuning Large Language Models (LLMs), with Proximal Policy Optimization (PPO) serving as the de facto standard algorith…
Defeating the Training-Inference Mismatch via FP16
Penghui Qi, Zichen Liu, Xiangxin Zhou +4
Reinforcement learning (RL) fine-tuning of large language models (LLMs) often suffers from instability due to the numerical mismatch between the training and inference policies. Wh…
Nonparametric Data Attribution for Diffusion Models
Yutian Zhao, Chao Du, Xiaosen Zheng +2
Data attribution for generative models seeks to quantify the influence of individual training examples on model outputs. Existing methods for diffusion models typically require acc…
Variational Reasoning for Language Models
Xiangxin Zhou, Zichen Liu, Haonan Wang +5
We introduce a variational reasoning framework for language models that treats thinking traces as latent variables and optimizes them through variational inference. Starting from t…
Language Models Can Learn from Verbal Feedback Without Scalar Rewards
Renjie Luo, Zichen Liu, Xiangyan Liu +5
LLMs are often trained with RL from human or AI feedback, yet such methods typically compress nuanced feedback into scalar rewards, discarding much of their richness and inducing s…
Continual Reinforcement Learning by Planning with Online World Models
Zichen Liu, Guoji Fu, Chao Du +2
Continual reinforcement learning (CRL) refers to a naturalistic setting where an agent needs to endlessly evolve, by trial and error, to solve multiple tasks that are presented seq…
QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Design
Benjamin Schneider, Dongfu Jiang, Chao Du +2
Long-video understanding has emerged as a crucial capability in real-world applications such as video surveillance, meeting summarization, educational lecture analysis, and sports…
BanditSpec: Adaptive Speculative Decoding via Bandit Algorithms
Yunlong Hou, Fengzhuo Zhang, Cunxiao Du +6
Speculative decoding has emerged as a popular method to accelerate the inference of Large Language Models (LLMs) while retaining their superior text generation performance. Previou…
Optimizing Anytime Reasoning via Budget Relative Policy Optimization
Penghui Qi, Zichen Liu, Tianyu Pang +3
Scaling test-time compute is crucial for enhancing the reasoning capabilities of large language models (LLMs). Existing approaches typically employ reinforcement learning (RL) to m…
Understanding R1-Zero-Like Training: A Critical Perspective
Zichen Liu, Changyu Chen, Wenjun Li +5
DeepSeek-R1-Zero has shown that reinforcement learning (RL) at scale can directly enhance the reasoning capabilities of LLMs without supervised fine-tuning. In this work, we critic…
Error Analyses of Auto-Regressive Video Diffusion Models: A Unified Framework
Jing Wang, Fengzhuo Zhang, Xiaoli Li +5
Auto-Regressive Video Diffusion Models (AR-VDMs) have shown strong capabilities in generating long, photorealistic videos, but suffer from two key limitations: (i) history forgetti…
LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification
Penghui Yang, Cunxiao Du, Fengzhuo Zhang +4
As Large Language Models (LLMs) can now process extremely long contexts, efficient inference over these extended inputs has become increasingly important, especially for emerging a…
Real-time Identity Defenses against Malicious Personalization of Diffusion Models
Hanzhong Guo, Shen Nie, Chao Du +3
Personalized generative diffusion models, capable of synthesizing highly realistic images based on a few reference portraits, may pose substantial social, ethical, and legal risks…
Improving Your Model Ranking on Chatbot Arena by Vote Rigging
Rui Min, Tianyu Pang, Chao Du +3
Chatbot Arena is a popular platform for evaluating LLMs by pairwise battles, where users vote for their preferred response from two randomly sampled anonymous models. While Chatbot…
Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Models
Zehan Wang, Ziang Zhang, Tianyu Pang +3
Orientation is a key attribute of objects, crucial for understanding their spatial pose and arrangement in images. However, practical solutions for accurate orientation estimation…
When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training
Haonan Wang, Qian Liu, Chao Du +4
Extending context window sizes allows large language models (LLMs) to process longer sequences and handle more complex tasks. Rotary Positional Embedding (RoPE) has become the de f…
Sample-Efficient Alignment for LLMs
Zichen Liu, Changyu Chen, Chao Du +2
We study methods for efficiently aligning large language models (LLMs) with human preferences given budgeted online feedback. We first formulate the LLM alignment problem in the fr…
Denial-of-Service Poisoning Attacks against Large Language Models
Kuofeng Gao, Tianyu Pang, Chao Du +3
Recent studies have shown that LLMs are vulnerable to denial-of-service (DoS) attacks, where adversarial inputs like spelling errors or non-semantic prompts trigger endless outputs…
Scaling up Masked Diffusion Models on Text
Shen Nie, Fengqi Zhu, Chao Du +5
Masked diffusion models (MDMs) have shown promise in language modeling, yet their scalability and effectiveness in core language tasks, such as text generation and language underst…
LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation
Xuan Zhang, Fengzhuo Zhang, Cunxiao Du +4
Scaling language models to handle longer contexts introduces substantial memory challenges due to the growing cost of key-value (KV) caches. Motivated by the efficiency gains of hy…
Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Concepts
Hongcheng Gao, Tianyu Pang, Chao Du +3
With the rapid progress of diffusion-based content generation, significant efforts are being made to unlearn harmful or copyrighted concepts from pretrained diffusion models (DMs)…
Improving Long-Text Alignment for Text-to-Image Diffusion Models
Luping Liu, Chao Du, Tianyu Pang +3
The rapid advancement of text-to-image (T2I) diffusion models has enabled them to generate unprecedented results from given texts. However, as text inputs become longer, existing e…
When Attention Sink Emerges in Language Models: An Empirical View
Xiangming Gu, Tianyu Pang, Chao Du +5
Language Models (LMs) assign significant attention to the first token, even if it is not semantically important, which is known as attention sink. This phenomenon has been widely a…