From the 1 of 11 linked papers with an AI index.
11 papers
Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning
Simeng Zhang, Yilong Chen, Wenyuan Zhang +4
Large language models often rely on Chain-of-Thought (CoT) reasoning to solve complex tasks, but verbose reasoning traces introduce substantial inference overhead. CoT compression…
Aggregating Visual Information with Optimal Transport for VideoLM Token Compression
Wenti Yin, Xiaotian Han, Junyuan Shang +5
Video language models process videos as dense visual-token sequences with substantial representational redundancy. Compressing these sequences is therefore essential for reducing t…
Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometry
Yehan Yang, Junyuan Shang, Yang Li +3
Long-context LLM inference is bottlenecked by quadratic attention computation and growing KV-cache costs. Existing sparse attention and KV-compression methods typically decide whic…
Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes
Tianyu Wang, Yuxuan Zhou, Wenbin Wang +3
The paper analyzes lossy verification techniques used in speculative decoding for large language models, categorizing them into truncation‑based and collaborative methods, identify…
ConSA: Controllable Sparsity in Hybrid Attention via Learnable Allocation
Yao Chen, Yinqi Yang, Junyuan Shang +6
Hybrid architectures combining full attention (FA) and sliding-window attention (SWA) are a promising paradigm for efficient LLM inference. However, existing methods typically rely…
Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Looping
Yao Chen, Yilong Chen, Yinqi Yang +9
Existing approaches to increasing the effective depth of Transformers predominantly rely on parameter reuse, extending computation through recursive execution. Under this paradigm,…