8 papers
AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregation
Chengyu Qiu, Xiao Fu, Fengcun Li +6
Attention--Feed-Forward Network (FFN) Disaggregation (AFD) is emerging as a promising architecture for serving Mixture-of-Experts (MoE) language models. While existing AFD systems…
LongCat Sparse Attention: Taming the Lightning via Streaming-aware Hierarchical Cross-Layer Indexing
Wen Zan, Jiaqi Zhang, Jianchao Tan +11
DeepSeek Sparse Attention (DSA) enables efficient long-context modeling through its Lightning Indexer. However, practical deployment remains constrained by the indexer's expensive…
Scaling Embeddings Outperforms Scaling Experts in Language Models
Hong Liu, Jiaqi Zhang, Chao Wang +13
While Mixture-of-Experts (MoE) architectures have become the standard for sparsity scaling in large language models, they increasingly face diminishing returns and system-level bot…
LongCat-Flash-Thinking-2601 Technical Report
Meituan LongCat Team, Anchun Gui, Bei Li +162
We introduce LongCat-Flash-Thinking-2601, a 560-billion-parameter open-source Mixture-of-Experts (MoE) reasoning model with superior agentic reasoning capability. LongCat-Flash-Thi…
Efficient Context Scaling with LongCat ZigZag Attention
Chen Zhang, Yang Bai, Jiahuan Li +19
We introduce LongCat ZigZag Attention (LoZA), which is a sparse attention scheme designed to transform any existing full-attention models into sparse versions with rather limited c…
Introducing LongCat-Flash-Thinking: A Technical Report
Meituan LongCat Team, Anchun Gui, Bei Li +122
We present LongCat-Flash-Thinking, an efficient 560-billion-parameter open-source Mixture-of-Experts (MoE) reasoning model. Its advanced capabilities are cultivated through a metic…