18 citations · 18 across the 3 of their papers we have counts for
3 papers
cs.LG2024
Collage: Light-Weight Low-Precision Strategy for LLM Training
Tao Yu, Gaurav Gupta, Karthick Gopalswamy +7
Large models training is plagued by the intense compute cost and limited hardware memory. A practical solution is low-precision representation but is troubled by loss in numerical…
cs.DC2018
Efficient Embedding of MPI Collectives in MXNET DAGs for scaling Deep Learning
Amith R Mamidala
Availability of high performance computing infrastructures such as clusters of GPUs and CPUs have fueled the growth of distributed learning systems. Deep Learning frameworks expres…
cs.DC2018★ 18 cited
MXNET-MPI: Embedding MPI parallelism in Parameter Server Task Model for scaling Deep Learning
Amith R Mamidala, Georgios Kollias, Chris Ward +1
Existing Deep Learning frameworks exclusively use either Parameter Server(PS) approach or MPI parallelism. In this paper, we discuss the drawbacks of such approaches and propose a…