5 papers · 1 filter
Information-Aware KV Cache Compression for Long Reasoning
Jushi Kai, Zhuiri Xiao, Alexandra Birch +1
Reasoning capability has advanced rapidly in large language models (LLMs), leading to an increasing size of key-value (KV) cache in both prefilling and decoding stages. Existing KV…
MLP Memory: A Retriever-Pretrained Memory for Large Language Models
Rubin Wei, Jiaqi Cao, Jiarui Wang +4
Modern approaches to enhancing Large Language Models' factual accuracy and knowledge utilization face a fundamental trade-off: non-parametric retrieval-augmented generation (RAG) p…
FreqKV: Key-Value Compression in Frequency Domain for Context Window Extension
Jushi Kai, Yixuan Wang, Boyi Zeng +4
Existing key-value (KV) cache compression methods for large language models (LLMs) often rely on token eviction, which risks losing critical local information in both long prefilli…
Leveraging Grammar Induction for Language Understanding and Generation
Jushi Kai, Shengyuan Hou, Yusheng Huang +1
Grammar induction has made significant progress in recent years. However, it is not clear how the application of induced grammar could enhance practical performance in downstream t…
Syntax-guided Localized Self-attention by Constituency Syntactic Distance
Shengyuan Hou, Jushi Kai, Haotian Xue +5
Recent works have revealed that Transformers are implicitly learning the syntactic information in its lower layers from data, albeit is highly dependent on the quality and scale of…