12 citations · 18 across the 5 of their papers we have counts for
5 papers
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
Zihan Qiu, Zekun Wang, Xiao Li +33
We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and additional 51B parameters of n…
Erase-then-Delta Attention: Decoupling Erase and Write Addresses in Delta-Rule Linear Attention
Xiao Li, Chengruidong Zhang, Hao Luo +15
Delta-rule linear attention improves recurrent memory updates by correcting what is already stored at the current write address before writing new content. However, the active corr…
VecAttention: Vector-wise Sparse Attention for Accelerating Long Context Inference
Anmin Liu, Ruixuan Yang, Huiqiang Jiang +5
Long-context video understanding and generation pose a significant computational challenge for Transformer-based video models due to the quadratic complexity of self-attention. Whi…
Qwen2.5-1M Technical Report
An Yang, Bowen Yu, Chengyuan Li +25
We introduce Qwen2.5-1M, a series of models that extend the context length to 1 million tokens. Compared to the previous 128K version, the Qwen2.5-1M series have significantly enha…
Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache
Bin Lin, Chen Zhang, Tao Peng +12
Large Language Models (LLMs) demonstrate substantial potential across a diverse array of domains via request serving. However, as trends continue to push for expanding context size…