20 citations · 51 across the 10 of their papers we have counts for
8 papers · 1 filter
Implicit Actions and Non-blocking Failure Recovery with MPI
Aurelien Bouteiller, George Bosilca
Scientific applications have long embraced the MPI as the environment of choice to execute on large distributed systems. The User-Level Failure Mitigation (ULFM) specification exte…
Callback-based Completion Notification using MPI Continuations
Joseph Schuchart, Philipp Samfass, Christoph Niethammer +2
Asynchronous programming models (APM) are gaining more and more traction, allowing applications to expose the available concurrency to a runtime system tasked with coordinating the…
Quo Vadis MPI RMA? Towards a More Efficient Use of MPI One-Sided Communication
Joseph Schuchart, Christoph Niethammer, José Gracia +1
The MPI standard has long included one-sided communication abstractions through the MPI Remote Memory Access (RMA) interface. Unfortunately, the MPI RMA chapter in the 4.0 version…
Task Bench: A Parameterized Benchmark for Evaluating Parallel Runtime Performance
Elliott Slaughter, Wei Wu, Yuankun Fu +12
We present Task Bench, a parameterized benchmark designed to explore the performance of parallel and distributed programming systems under a variety of application scenarios. Task…
SuperNeurons: FFT-based Gradient Sparsification in the Distributed Training of Deep Neural Networks
Linnan Wang, Wei Wu, Junyu Zhang +4
The performance and efficiency of distributed training of Deep Neural Networks highly depend on the performance of gradient averaging among all participating nodes, which is bounde…
Efficient Communications in Training Large Scale Neural Networks
Linnan Wang, Wei Wu, George Bosilca +2
We consider the problem of how to reduce the cost of communication that is required for the parallel training of a neural network. The state-of-the-art method, Bulk Synchronous Par…