6 papers
Voltage-to-temperature calibration of the High Altitude THz Solar telescope acquisition system
Gedeane G. S. Kenshima, Daniel R. Sousa, Tiago Giorgetti +2
The THz range has been under-explored for solar astronomy, mainly due to technological limitations. Only recently, a few new telescopes, such as the High Altitude Terahertz Solar (…
UniMoMo: Expert Merging-Based MoE Acceleration for Large Recommendation Models
Lei Xin, Bin Gu, Peize Li +9
Sparse mixture-of-experts (MoE) layers expand recommendation capacity through conditional computation, yet a trained checkpoint still stores and routes over its full expert bank. W…
Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation
Zunhai Su, Hengyuan Zhang, Wei Wu +24
As the foundational architecture of modern machine learning, Transformers have driven remarkable progress across diverse AI domains. Despite their transformative impact, a persiste…
OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
Zunhai Su, Rui Yang, Chao Zhang +11
The rapid advancement toward long-context reasoning and multi-modal intelligence has made the memory footprint of the Key-Value (KV) cache a dominant memory bottleneck for efficien…
SnapMLA: Efficient Long-Context MLA Decoding via Hardware-Aware FP8 Quantized Pipelining
Yifan Zhang, Zunhai Su, Shuhao Hu +5
While FP8 attention has shown substantial promise in innovations like FlashAttention-3, its integration into the decoding phase of the DeepSeek Multi-head Latent Attention (MLA) ar…
LongCat-Flash Technical Report
Meituan LongCat Team, Bayan, Bei Li +179
We introduce LongCat-Flash, a 560-billion-parameter Mixture-of-Experts (MoE) language model designed for both computational efficiency and advanced agentic capabilities. Stemming f…