Clairvoyant Prefetching for Distributed Machine Learning I/O

Nikoli Dryden, Roman Böhringer, Tal Ben-Nun +1

I/O is emerging as a major bottleneck for machine learning training, especially in distributed environments. Indeed, at large scale, I/O takes as much as 85% of training time. Addr…