1 citations · 1 across the 15 of their papers we have counts for
7 papers · 1 filter
AsyncTLS: Efficient Generative LLM Inference with Asynchronous Two-level Sparse Attention
Yuxuan Hu, Jianchao Tan, Jiaqi Zhang +7
Long-context inference in LLMs faces the dual challenges of quadratic attention complexity and prohibitive KV cache memory. While token-level sparse attention offers superior accur…
Accelerate Speculative Decoding with Sparse Computation in Verification
Jikai Wang, Jianchao Tan, Yuxuan Hu +6
Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel. However, the verification stage often becomes the dominant…
Optimizing Native Sparse Attention with Latent Attention and Local Global Alternating Strategies
Yuxuan Hu, Jianchao Tan, Jiaqi Zhang +7
In this work, we conduct a systematic analysis of Native Sparse Attention (NSA) and propose targeted improvements that enhance long-context modeling. A key insight is that alternat…
LongCat-Flash Technical Report
Meituan LongCat Team, Bayan, Bei Li +179
We introduce LongCat-Flash, a 560-billion-parameter Mixture-of-Experts (MoE) language model designed for both computational efficiency and advanced agentic capabilities. Stemming f…
MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
Jiayu Qin, Jianchao Tan, Kefeng Zhang +2
The remarkable performance of large language models (LLMs) in various language tasks has attracted considerable attention. However, the ever-increasing size of these models present…
C2T: A Classifier-Based Tree Construction Method in Speculative Decoding
Feiye Huo, Jianchao Tan, Kefeng Zhang +2
The growing scale of Large Language Models (LLMs) has exacerbated inference latency and computational costs. Speculative decoding methods, which aim to mitigate these issues, often…
EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
Yulei Qian, Fengcun Li, Xiangyang Ji +4
The Mixture-of-Experts (MoE) model has emerged as a prominent architecture in the field of Large Language Models (LLMs), providing a better balance between model performance and co…