distributed cache 1hybrid sliding window attention 1kvcache optimization 1mixture-of-experts 1multimodal inference 1
From the 1 of 2 linked papers with an AI index.
2 papers
cs.AR2026
Full-Pipeline Inference Optimization for MiMo-V2.5 Series: Pushing Hybrid SWA Efficiency to the Limit
Xiaomi MiMo Team, Anqi Liu, Aoxin Ma +28
The paper describes a production-ready inference system for the MiMo-V2.5 large language model family that combines hybrid sliding window attention, sparse mixture-of-experts, and…
cs.CL2026
HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing
Yizhao Gao, Jianyu Wei, Qihao Zhang +11
This work introduces Hybrid Sparse Attention (HySparse), a new architecture that interleaves each full attention layer with several sparse attention layers. While conceptually simp…