9 papers
HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps
Xin Liu, Runsong Zhao, Xinyu Liu +8
Extended chain-of-thought (CoT) traces improve LLM reasoning but incur substantial computational and memory costs. While existing CoT compression methods mitigate this by condensin…
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
Xinyu Liu, Kechen Jiao, Chunyang Xiao +10
On-policy distillation (OPD) has become a promising paradigm for reasoning-oriented post-training of large language models (LLMs), especially when combined with reinforcement learn…
MemoSight: Unifying Context Compression and Multi Token Prediction for Reasoning Acceleration
Xinyu Liu, Xin Liu, Bo Jin +8
While chain-of-thought (CoT) reasoning enables LLMs to solve challenging reasoning tasks, the linear growth of the KV cache leads to substantial memory and inference overhead. Exis…
CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling
Runsong Zhao, Shilei Liu, Jiwei Tang +8
The quadratic complexity and indefinitely growing key-value (KV) cache of standard Transformers pose a major barrier to long-context processing. To overcome this, we introduce the…
PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction
Runsong Zhao, Shilei Liu, Jiwei Tang +8
While context compression can mitigate the growing inference costs of Large Language Models (LLMs) by shortening contexts, existing methods that specify a target compression ratio…
Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors
Xin Liu, Runsong Zhao, Pengcheng Huang +7
Context compression is an advanced technique that accelerates large language model (LLM) inference by converting long inputs into compact representations. Existing methods primaril…