6 papers · 1 filter
AsyncTLS: Efficient Generative LLM Inference with Asynchronous Two-level Sparse Attention
Yuxuan Hu, Jianchao Tan, Jiaqi Zhang +7
Long-context inference in LLMs faces the dual challenges of quadratic attention complexity and prohibitive KV cache memory. While token-level sparse attention offers superior accur…
Scaling Embeddings Outperforms Scaling Experts in Language Models
Hong Liu, Jiaqi Zhang, Chao Wang +13
While Mixture-of-Experts (MoE) architectures have become the standard for sparsity scaling in large language models, they increasingly face diminishing returns and system-level bot…
Efficient Context Scaling with LongCat ZigZag Attention
Chen Zhang, Yang Bai, Jiahuan Li +19
We introduce LongCat ZigZag Attention (LoZA), which is a sparse attention scheme designed to transform any existing full-attention models into sparse versions with rather limited c…
Accelerate Speculative Decoding with Sparse Computation in Verification
Jikai Wang, Jianchao Tan, Yuxuan Hu +6
Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel. However, the verification stage often becomes the dominant…
Optimizing Native Sparse Attention with Latent Attention and Local Global Alternating Strategies
Yuxuan Hu, Jianchao Tan, Jiaqi Zhang +7
In this work, we conduct a systematic analysis of Native Sparse Attention (NSA) and propose targeted improvements that enhance long-context modeling. A key insight is that alternat…
LongCat-Flash Technical Report
Meituan LongCat Team, Bayan, Bei Li +179
We introduce LongCat-Flash, a 560-billion-parameter Mixture-of-Experts (MoE) language model designed for both computational efficiency and advanced agentic capabilities. Stemming f…