5 papers
Mechanism-Aligned Observability for Training Instability of Large Language Models
Ruixuan Huang, Yipei Wang, Hantao Huang +7
Frontier large language model training consumes massive accelerator fleets over long wall-clock times, making stability failures extremely costly. However, global symptoms such as…
Continual LLM Upcycling: A Predictor-Gated Bank-Wise Sparsity Training Recipe for Dense-to-Sparse LLMs
Ruixuan Huang, Jinyuan Shi, Hantao Huang +5
We study dense-to-sparse continual training as a way to construct channel-sparse large language models from dense checkpoints. Starting from a Qwen2.5-8B dense backbone, we continu…
SQ-format: A Unified Sparse-Quantized Hardware-friendly Data Format for LLMs
Ruixuan Huang, Hao Zeng, Hantao Huang +4
Post-training quantization (PTQ) plays a crucial role in the democratization of large language models (LLMs). However, existing low-bit quantization and sparsification techniques a…
RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress
Ruixuan Huang, Qingyue Wang, Hantao Huang +4
Mixture-of-Experts architectures have become the standard for scaling large language models due to their superior parameter efficiency. To accommodate the growing number of experts…
SALS: Sparse Attention in Latent Space for KV cache Compression
Junlin Mu, Hantao Huang, Jihang Zhang +3
Large Language Models capable of handling extended contexts are in high demand, yet their inference remains challenging due to substantial Key-Value cache size and high memory band…