3 papers
cs.CL2025
LooGLE v2: Are LLMs Ready for Real World Long Dependency Challenges?
Ziyuan He, Yuxuan Wang, Jiaqi Li +2
Large language models (LLMs) are equipped with increasingly extended context windows recently, yet their long context understanding capabilities over long dependency tasks remain f…
cs.LG2025
TPLA: Tensor Parallel Latent Attention for Efficient Disaggregated Prefill and Decode Inference
Xiaojuan Tang, Fanxu Meng, Pingzhi Tang +4
Multi-Head Latent Attention (MLA), introduced in DeepSeek-V2, compresses key-value states into a low-rank latent vector, caching only this vector to reduce memory. In tensor parall…
cs.CL2024
LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning
Yansheng Mao, Jiaqi Li, Fanxu Meng +3
Long context understanding remains challenging for large language models due to their limited context windows. This paper introduces Long Input Fine-Tuning (LIFT) for long context…