2 papers
cs.AI2026
AttentionIndex: Accelerating Learned Sparse Attention with a Causal Inverted-File Index
Jeongmin Brian Park, Wen-mei Hwu
Learned sparse attention, used in production models such as DeepSeek-V4 and GLM-5, lets each query attend to only k keys chosen by a lightweight indexer, but the indexer still scor…
cs.DC2024
SSDTrain: An Activation Offloading Framework to SSDs for Faster Large Language Model Training
Kun Wu, Jeongmin Brian Park, Xiaofan Zhang +5
The growth rate of the GPU memory capacity has not been able to keep up with that of the size of large language models (LLMs), hindering the model training process. In particular,…