Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024
Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
Yong Liu, Zirui Zhu, Chaoyu Gong +3
While fine-tuning large language models (LLMs) for specific tasks often yields impressive results, it comes at the cost of memory inefficiency due to back-propagation in gradient-b…
cs.LG2021
An Efficient 2D Method for Training Super-Large Deep Learning Models
Qifan Xu, Shenggui Li, Chaoyu Gong +1
Huge neural network models have shown unprecedented performance in real-world applications. However, due to memory constraints, model parallelism must be utilized to host large mod…