1 citations · 1 across the 5 of their papers we have counts for
5 papers
Mechanism-Aligned Observability for Training Instability of Large Language Models
Ruixuan Huang, Yipei Wang, Hantao Huang +7
Frontier large language model training consumes massive accelerator fleets over long wall-clock times, making stability failures extremely costly. However, global symptoms such as…
Continual LLM Upcycling: A Predictor-Gated Bank-Wise Sparsity Training Recipe for Dense-to-Sparse LLMs
Ruixuan Huang, Jinyuan Shi, Hantao Huang +5
We study dense-to-sparse continual training as a way to construct channel-sparse large language models from dense checkpoints. Starting from a Qwen2.5-8B dense backbone, we continu…
AsyncHZP: Hierarchical ZeRO Parallelism with Asynchronous Scheduling for Scalable LLM Training
Huawei Bai, Yifan Huang, Wenqi Shi +4
The training efficiency and scalability of language models on massive clusters currently remain a critical bottleneck. Mainstream approaches like ND parallelism are often cumbersom…
Seed-Coder: Let the Code Model Curate Data for Itself
ByteDance Seed, Yuyu Zhang, Jing Su +24
Code data in large language model (LLM) pretraining is recognized crucial not only for code-related tasks but also for enhancing general intelligence of LLMs. Current open-source L…
A Vulnerability Code Intent Summary Dataset
Yifan Huang, Weisong Sun, Yubin Qu
In the era of Large Language Models (LLMs), the code summarization technique boosts a lot, along with the emergence of many new significant works. However, the potential of code su…