A Distribution-Matching Perspective on LLM Training Algorithms

Yanxing Huang, Bingzhang Yan, Zihan Tang +3

In this work, we propose a unified framework for training algorithms across different stages of modern large language model development, based on the perspective of distribution ma…