3 papers
cs.LG2026
FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention
Yan Wang, Qifan Zhang, Jiachen Yu +12
Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose \textbf{Lookahead…
cs.LG2026
Breaking the Tokenizer Barrier: On-Policy Distillation across Model Families
Yifan Niu, Han Xiao, Dongyi Liu +4
On-Policy Distillation (OPD) has become a core technique in the post-training of Large Language Models (LLMs) for transferring knowledge from domain experts to student models. Howe…
cs.CL2024
Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations
Nuo Chen, Zinan Zheng, Ning Wu +3
Existing research predominantly focuses on developing powerful language learning models (LLMs) for mathematical reasoning within monolingual languages, with few explorations in pre…