2 citations · 2 across the 2 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
Yuxi Liu, Renjia Deng, Yutong He +3
The substantial memory demands of pre-training and fine-tuning large language models (LLMs) require memory-efficient optimization algorithms. One promising approach is layer-wise o…
cs.LG2024★ 2 cited
SAttention: Improving Long Sequence Attention with Smoothed Skeleton Sketching
Xue Wang, Tian Zhou, Jianqing Zhu +6
Attention based models have achieved many remarkable breakthroughs in numerous applications. However, the quadratic complexity of Attention makes the vanilla Attention based models…