4 papers · 1 filter
Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
Dongwon Jo, Beomseok Kang, Jiwon Song +1
The quadratic complexity of attention remains the central bottleneck in long-context inference for large language models. Prior acceleration methods either sparsify the attention m…
Retrospective Sparse Attention for Efficient Long-Context Generation
Seonghwan Choi, Beomseok Kang, Dongwon Jo +1
Large Language Models (LLMs) are increasingly deployed in long-context tasks such as reasoning, code generation, and multi-turn dialogue. However, inference over extended contexts…
QWHA: Quantization-Aware Walsh-Hadamard Adaptation for Parameter-Efficient Fine-Tuning on Large Language Models
Hyesung Jeon, Seojune Lee, Beomseok Kang +2
The demand for efficient deployment of large language models (LLMs) has driven interest in quantization, which reduces inference cost, and parameter-efficient fine-tuning (PEFT), w…
LiteStage: Latency-aware Layer Skipping for Multi-stage Reasoning
Beomseok Kang, Jiwon Song, Jae-Joon Kim
Multi-stage reasoning has emerged as an effective strategy for enhancing the reasoning capability of small language models by decomposing complex problems into sequential sub-stage…