22 citations · 43 across the 9 of their papers we have counts for
9 papers
Tightening I/O Lower Bounds through the Hourglass Dependency Pattern
Lionel Eyraud-Dubois, Guillaume Iooss, Julien Langou +1
When designing an algorithm, one cares about arithmetic/computational complexity, but data movement (I/O) complexity plays an increasingly important role that highly impacts perfor…
Survey on Large Scale Neural Network Training
Julia Gusak, Daria Cherniuk, Alena Shilova +8
Modern Deep Neural Networks (DNNs) require significant memory to store weight, activations, and other intermediate tensors during training. Hence, many models do not fit one GPU de…
I/O-Optimal Algorithms for Symmetric Linear Algebra Kernels
Olivier Beaumont, Lionel Eyraud-Dubois, Mathieu Vérité +1
In this paper, we consider two fundamental symmetric kernels in linear algebra: the Cholesky factorization and the symmetric rank- update (SYRK), with the classical three nested…
Optimal checkpointing for heterogeneous chains: how to train deep neural networks with limited memory
Julien Herrmann, Olivier Beaumont, Lionel Eyraud-Dubois +3
This paper introduces a new activation checkpointing method which allows to significantly decrease memory usage when training Deep Neural Networks with the back-propagation algorit…
Scheduling on Two Types of Resources: a Survey
Olivier Beaumont, Louis-claude Canon, Lionel Eyraud-Dubois +5
The evolution in the design of modern parallel platforms leads to revisit the scheduling jobs on distributed heterogeneous resources. The goal of this survey is to present the main…
Performance Models for Data Transfers: A Case Study with Molecular Chemistry Kernels
Suraj Kumar, Lionel Eyraud-Dubois, Sriram Krishnamoorthy
With increasing complexity of hardwares, systems with different memory nodes are ubiquitous in High Performance Computing (HPC). It is paramount to develop strategies to overlap th…