1 citations · 2 across the 7 of their papers we have counts for
9 papers
Robust LLM Training Infrastructure at ByteDance
Borui Wan, Gaohong Liu, Zuquan Song +32
The training scale of large language models (LLMs) has reached tens of thousands of GPUs and is still continuously expanding, enabling faster learning of larger models. Accompanyin…
Mycroft: Tracing Dependencies in Collective Communication Towards Reliable LLM Training
Yangtao Deng, Lei Zhang, Qinlong Wang +13
Reliability is essential for ensuring efficiency in LLM training. However, many real-world reliability issues remain difficult to resolve, resulting in wasted resources and degrade…
Balanced Actor Initialization: Stable RLHF Training of Distillation-Based Reasoning Models
Chen Zheng, Yiyuan Ma, Yuan Yang +11
The development of alignment and reasoning capabilities in large language models has seen remarkable progress through two paradigms: instruction tuning and reinforcement learning f…
BootSeer: Analyzing and Mitigating Initialization Bottlenecks in Large-Scale LLM Training
Rui Li, Xiaoyun Zhi, Jinxin Chi +14
Large Language Models (LLMs) have become a cornerstone of modern AI, driving breakthroughs in natural language processing and expanding into multimodal jobs involving images, audio…
Understanding Stragglers in Large Model Training Using What-if Analysis
Jinkun Lin, Ziheng Jiang, Zuquan Song +13
Large language model (LLM) training is one of the most demanding distributed computations today, often requiring thousands of GPUs with frequent synchronization across machines. Su…
Seed1.5-VL Technical Report
Dong Guo, Faming Wu, Feida Zhu +194
We present Seed1.5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning. Seed1.5-VL is composed with a 532M-parameter v…