DISTREAL: Distributed Resource-Aware Learning in Heterogeneous Systems

Martin Rapp, Ramin Khalili, Kilian Pfeiffer +1

We study the problem of distributed training of neural networks (NNs) on devices with heterogeneous, limited, and time-varying availability of computational resources. We present a…