6 papers
HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps
Xin Liu, Runsong Zhao, Xinyu Liu +8
Extended chain-of-thought (CoT) traces improve LLM reasoning but incur substantial computational and memory costs. While existing CoT compression methods mitigate this by condensin…
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
Xinyu Liu, Kechen Jiao, Chunyang Xiao +10
On-policy distillation (OPD) has become a promising paradigm for reasoning-oriented post-training of large language models (LLMs), especially when combined with reinforcement learn…
MemoSight: Unifying Context Compression and Multi Token Prediction for Reasoning Acceleration
Xinyu Liu, Xin Liu, Bo Jin +8
While chain-of-thought (CoT) reasoning enables LLMs to solve challenging reasoning tasks, the linear growth of the KV cache leads to substantial memory and inference overhead. Exis…
Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors
Xin Liu, Runsong Zhao, Pengcheng Huang +7
Context compression is an advanced technique that accelerates large language model (LLM) inference by converting long inputs into compact representations. Existing methods primaril…
Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models
Runsong Zhao, Xin Liu, Xinyu Liu +4
Using special tokens (e.g., gist, memory, or compressed tokens) to compress context information is a common practice for large language models (LLMs). However, existing approaches…
Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-context Models
Xinyu Liu, Runsong Zhao, Pengcheng Huang +5
Numerous recent works target to extend effective context length for language models and various methods, tasks and benchmarks exist to measure model's effective memorization length…