6 papers
Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention
Wenhu Zhang, Yiming Wu, Huanyu Wang +6
Diffusion Language Models (DLMs) enable globally coherent, bidirectional, and controllable text generation, offering advantages over traditional autoregressive LLMs, while scaling…
Convex Dataset Valuation for Post-Training
Siqi Zeng, Christopher Jung, Rui Li +7
Improving LLM performance on downstream tasks sometimes requires leveraging auxiliary datasets during post-training. In practice, however, developers face constraints on compute, l…
Expectation Error Bounds for Transfer Learning in Linear Regression and Linear Neural Networks
Meitong Liu, Christopher Jung, Rui Li +2
In transfer learning, the learner leverages auxiliary data to improve generalization on a main task. However, the precise theoretical understanding of when and how auxiliary data h…
Gradual Domain Adaptation: Theory and Algorithms
Yifei He, Haoxiang Wang, Bo Li +1
Unsupervised domain adaptation (UDA) adapts a model from a labeled source domain to an unlabeled target domain in a one-off way. Though widely applied, UDA faces a great challenge…
RLHF Workflow: From Reward Modeling to Online RLHF
Hanze Dong, Wei Xiong, Bo Pang +7
We present the workflow of Online Iterative Reinforcement Learning from Human Feedback (RLHF) in this technical report, which is widely reported to outperform its offline counterpa…
Mitigating the Alignment Tax of RLHF
Yong Lin, Hangyu Lin, Wei Xiong +14
LLMs acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, w…