2 citations · 3 across the 8 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.LG2024
A Convex-optimization-based Layer-wise Post-training Pruner for Large Language Models
Pengxiang Zhao, Hanyu Hu, Ping Li +3
Pruning is a critical strategy for compressing trained large language models (LLMs), aiming at substantial memory conservation and computational acceleration without compromising p…
cs.DC2024★ 1 cited
Optimizing Large Model Training through Overlapped Activation Recomputation
Ping Chen, Wenjie Zhang, Shuibing He +10
Large model training often uses recomputation to alleviate memory pressure and pipelines to exploit the parallelism of data, tensors, and devices. However, existing recomputation a…
cs.LG2024★ 2 cited
Adapprox: Adaptive Approximation in Adam Optimization via Randomized Low-Rank Matrices
Pengxiang Zhao, Ping Li, Yingjie Gu +4
As deep learning models exponentially increase in size, optimizers such as Adam encounter significant memory consumption challenges due to the storage of first and second moment da…