7 papers
Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
Dongwon Jo, Beomseok Kang, Jiwon Song +1
The quadratic complexity of attention remains the central bottleneck in long-context inference for large language models. Prior acceleration methods either sparsify the attention m…
Retrospective Sparse Attention for Efficient Long-Context Generation
Seonghwan Choi, Beomseok Kang, Dongwon Jo +1
Large Language Models (LLMs) are increasingly deployed in long-context tasks such as reasoning, code generation, and multi-turn dialogue. However, inference over extended contexts…
Rotation-Aligned Key Channel Pruning for Efficient Vision-Language Model Inference
Beomseok Kang, Dongwon Jo, Jiwon Song +2
Vision-Language Models suffer severe KV cache pressure at inference, as a single image often encodes into thousands of tokens. Most existing methods exploit token sparsity through…
CompactAttention: Accelerating Chunked Prefill with Block-Union KV Selection
Jiwon Song, Dongwon Jo, Beomseok Kang +1
Chunked prefill has become a widely adopted serving strategy for long-context large language models, but efficient attention computation in this regime remains challenging. Existin…
FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Acceleration
Dongwon Jo, Jiwon Song, Yulhwa Kim +1
While large language models (LLMs) excel at handling long-context sequences, they require substantial prefill computation and key-value (KV) cache, which can heavily burden computa…
Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning
Jiwon Song, Dongwon Jo, Yulhwa Kim +1
Recent reasoning-focused language models achieve high accuracy by generating lengthy intermediate reasoning paths before producing final answers. While this approach is effective i…