5 citations · 6 across the 7 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
Jongseok Park, Sunga Kim, Zhenyu Gu +2
Mixture of Experts (MoE) architecture has become the standard for state-of-the-art large language models, owing to its computational efficiency through sparse expert activation. Ho…
cs.LG2023
An Evaluation of Memory Optimization Methods for Training Neural Networks
Xiaoxuan Liu, Siddharth Jha, Alvin Cheung
As models continue to grow in size, the development of memory optimization methods (MOMs) has emerged as a solution to address the memory bottleneck encountered when training large…
cs.LG2022★ 5 cited
GACT: Activation Compressed Training for Generic Network Architectures
Xiaoxuan Liu, Lianmin Zheng, Dequan Wang +9
Training large neural network (NN) models requires extensive memory resources, and Activation Compressed Training (ACT) is a promising approach to reduce training memory footprint.…