Showing cs.CLShow all
3 papers · 1 filter
cs.CL2026
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
Zihan Qiu, Zekun Wang, Xiao Li +33
We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and additional 51B parameters of n…
cs.CL2026
H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inference
Hao Yu, Zheng Li, Dayiheng Liu +1
The NVIDIA Blackwell architecture, with native support for the ultra-fine-grained NVFP4 format, opens new opportunities for accelerating large language model (LLM) inference. NVFP4…
cs.CL2026
A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training
Zihan Qiu, Zeyu Huang, Kaiyue Wen +16
We investigate the functional role of emergent outliers in large language models, specifically attention sinks (a few tokens that consistently receive large attention logits) and r…