3 citations · 3 across the 6 of their papers we have counts for
4 papers · 1 filter
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
Zihan Qiu, Zekun Wang, Xiao Li +33
We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and additional 51B parameters of n…
Libra: Taming Attention Workload Skew in Long-Context LLM Training with Bounded Sequence Pool
Yan Wang, Xiulong Yuan, Kaiming Yang +16
Long-context LLM training suffers from a load-balancing problem that sequence packing does not solve. Packing samples into fixed-token sequences balances memory and linear-cost ope…
Erase-then-Delta Attention: Decoupling Erase and Write Addresses in Delta-Rule Linear Attention
Xiao Li, Chengruidong Zhang, Hao Luo +15
Delta-rule linear attention improves recurrent memory updates by correcting what is already stored at the current write address before writing new content. However, the active corr…
Accelerating Compound LLM Training Workloads with Maestro
Xiulong Yuan, Hongqing Chen, Jiaxuan Peng +16
Compound LLM training workloads-such as knowledge distillation and multimodal LLM (MLLM) training-are gaining prominence. These typically comprise heterogeneous components differin…