4 papers
100-LongBench: Are de facto Long-Context Benchmarks Literally Evaluating Long-Context Ability?
Wang Yang, Hongye Jin, Shaochen Zhong +4
Long-context capability is considered one of the most important abilities of LLMs, as a truly long context-capable LLM enables users to effortlessly process many originally exhaust…
Quantize What Counts: More for Keys, Less for Values
Mohsen Hariri, Alan Luo, Weicong Chen +6
Large Language Models (LLMs) suffer inference-time memory bottlenecks dominated by the attention Key-Value (KV) cache, which scales with model size and context length. While KV-cac…
Demystifying Hybrid Thinking: Can LLMs Truly Switch Between Think and No-Think?
Shouren Wang, Wang Yang, Xianxuan Long +3
Hybrid thinking enables LLMs to switch between reasoning and direct answering, offering a balance between efficiency and reasoning capability. Yet our experiments reveal that curre…
SELF: Self-Extend the Context Length With Logistic Growth Function
Phat Thanh Dang, Saahil Thoppay, Wang Yang +3
Large language models suffer issues when operated on long contexts that are larger than their training context length due to the standard position encoding for tokens in the attent…