5 papers
Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
Dongwon Jo, Beomseok Kang, Jiwon Song +1
The quadratic complexity of attention remains the central bottleneck in long-context inference for large language models. Prior acceleration methods either sparsify the attention m…
Retrospective Sparse Attention for Efficient Long-Context Generation
Seonghwan Choi, Beomseok Kang, Dongwon Jo +1
Large Language Models (LLMs) are increasingly deployed in long-context tasks such as reasoning, code generation, and multi-turn dialogue. However, inference over extended contexts…
QWHA: Quantization-Aware Walsh-Hadamard Adaptation for Parameter-Efficient Fine-Tuning on Large Language Models
Hyesung Jeon, Seojune Lee, Beomseok Kang +2
The demand for efficient deployment of large language models (LLMs) has driven interest in quantization, which reduces inference cost, and parameter-efficient fine-tuning (PEFT), w…
LiteStage: Latency-aware Layer Skipping for Multi-stage Reasoning
Beomseok Kang, Jiwon Song, Jae-Joon Kim
Multi-stage reasoning has emerged as an effective strategy for enhancing the reasoning capability of small language models by decomposing complex problems into sequential sub-stage…
OrbitAll: A Unified Quantum Mechanical Representation Deep Learning Framework for All Molecular Systems
Beom Seok Kang, Vignesh C. Bhethanabotla, Amin Tavakoli +6
We introduce OrbitAll, a geometry- and physics-informed deep learning framework that encodes any molecular system with arbitrary charges, spins, and environmental effects using ele…