MCR-DL: Mix-and-Match Communication Runtime for Deep Learning

Quentin Anthony, Ammar Ahmad Awan, Jeff Rasley +5

In recent years, the training requirements of many state-of-the-art Deep Learning (DL) models have scaled beyond the compute and memory capabilities of a single processor, and nece…