21 citations · 22 across the 2 of their papers we have counts for
2 papers
cs.DC2022★ 1 cited
Elixir: Train a Large Language Model on a Small GPU Cluster
Haichen Huang, Jiarui Fang, Hongxin Liu +2
In recent years, large language models have achieved great success due to their unprecedented size. However, training these models poses a challenge for most researchers as it requ…
cs.LG2021★ 21 cited
Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training
Shenggui Li, Hongxin Liu, Zhengda Bian +5
The success of Transformer models has pushed the deep learning model scale to billions of parameters. Due to the limited memory resource of a single GPU, However, the best practice…