3 citations · 6 across the 9 of their papers we have counts for
10 papers
GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injection
Chaobo Jia, Ruipeng Wan, Ting Sun +5
LLM-based game generation promises to turn natural-language specifications into executable games, but progress is limited by the lack of reliable automated verification. Unlike con…
Robust LLM Training Infrastructure at ByteDance
Borui Wan, Gaohong Liu, Zuquan Song +32
The training scale of large language models (LLMs) has reached tens of thousands of GPUs and is still continuously expanding, enabling faster learning of larger models. Accompanyin…
Laminar: A Scalable Asynchronous RL Post-Training Framework
Guangming Sheng, Yuxuan Tong, Borui Wan +10
Reinforcement learning (RL) post-training for Large Language Models (LLMs) is now scaling to large clusters and running for extended durations to enhance model reasoning performanc…
Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning
ByteDance Seed, :, Jiaze Chen +267
We introduce Seed1.5-Thinking, capable of reasoning through thinking before responding, resulting in improved performance on a wide range of benchmarks. Seed1.5-Thinking achieves 8…
Efficient LLM Serving on Hybrid Real-time and Best-effort Requests
Wan Borui, Zhao Juntao, Jiang Chenyu +2
Recent breakthroughs in large Language Models (LLMs) have enabled various generative tasks on a single model. Real-world services (e.g., OpenAI's ChatGPT [27]) powered by an LLM of…
MegaScale-Data: Scaling Dataloader for Multisource Large Foundation Model Training
Juntao Zhao, Qi Lu, Wei Jia +13
Modern frameworks for training large foundation models (LFMs) employ dataloaders in a data-parallel manner, with each loader processing a disjoint subset of training data. When pre…
QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices
Juntao Zhao, Borui Wan, Yanghua Peng +3
A number of production deep learning clusters have attempted to explore inference hardware for DNN training, at the off-peak serving hours with many inference GPUs idling. Conducti…
ByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Development
Borui Wan, Mingji Han, Yiyao Sheng +9
Checkpointing to preserve training states is crucial during the development of Large Foundation Models (LFMs), for training resumption upon various failures or changes in GPU resou…
LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization
Juntao Zhao, Borui Wan, Yanghua Peng +2
Recent breakthroughs in Large-scale language models (LLMs) have demonstrated impressive performance on various tasks. The immense sizes of LLMs have led to very high resource deman…
Adaptive Message Quantization and Parallelization for Distributed Full-graph GNN Training
Borui Wan, Juntao Zhao, Chuan Wu
Distributed full-graph training of Graph Neural Networks (GNNs) over large graphs is bandwidth-demanding and time-consuming. Frequent exchanges of node features, embeddings and emb…