22 citations · 50 across the 10 of their papers we have counts for
10 papers
Rockmate: an Efficient, Fast, Automatic and Generic Tool for Re-materialization in PyTorch
Xunyi Zhao, Théotime Le Hellard, Lionel Eyraud +2
We propose Rockmate to control the memory requirements when training PyTorch DNN models. Rockmate is an automatic tool that starts from the model code and generates an equivalent m…
Survey on Large Scale Neural Network Training
Julia Gusak, Daria Cherniuk, Alena Shilova +8
Modern Deep Neural Networks (DNNs) require significant memory to store weight, activations, and other intermediate tensors during training. Hence, many models do not fit one GPU de…
I/O-Optimal Algorithms for Symmetric Linear Algebra Kernels
Olivier Beaumont, Lionel Eyraud-Dubois, Mathieu Vérité +1
In this paper, we consider two fundamental symmetric kernels in linear algebra: the Cholesky factorization and the symmetric rank- update (SYRK), with the classical three nested…
Geometric Deep Reinforcement Learning for Dynamic DAG Scheduling
Nathan Grinsztajn, Olivier Beaumont, Emmanuel Jeannot +1
In practice, it is quite common to face combinatorial optimization problems which contain uncertainty along with non-determinism and dynamicity. These three properties call for app…
Optimal checkpointing for heterogeneous chains: how to train deep neural networks with limited memory
Julien Herrmann, Olivier Beaumont, Lionel Eyraud-Dubois +3
This paper introduces a new activation checkpointing method which allows to significantly decrease memory usage when training Deep Neural Networks with the back-propagation algorit…
Scheduling on Two Types of Resources: a Survey
Olivier Beaumont, Louis-claude Canon, Lionel Eyraud-Dubois +5
The evolution in the design of modern parallel platforms leads to revisit the scheduling jobs on distributed heterogeneous resources. The goal of this survey is to present the main…
Training on the Edge: The why and the how
Navjot Kukreja, Alena Shilova, Olivier Beaumont +4
Edge computing is the natural progression from Cloud computing, where, instead of collecting all data and processing it centrally, like in a cloud computing environment, we distrib…
Analysis of Dynamic Scheduling Strategies for Matrix Multiplication on Heterogeneous Platforms
Olivier Beaumont, Loris Marchal
The tremendous increase in the size and heterogeneity of supercomputers makes it very difficult to predict the performance of a scheduling algorithm. Therefore, dynamic solutions,…
Efficient and Robust Allocation Algorithms in Clouds under Memory Constraints
Olivier Beaumont, Lionel Eyraud-Dubois, Paul Renaud-Goud
We consider robust resource allocation of services in Clouds. More specifically, we consider the case of a large public or private Cloud platform that runs a relatively small set o…
Approximation Algorithms for Energy Minimization in Cloud Service Allocation under Reliability Constraints
Olivier Beaumont, Philippe Duchon, Paul Renaud-Goud
We consider allocation problems that arise in the context of service allocation in Clouds. More specifically, we assume on the one part that each computing resource is associated t…