37 citations · 116 across the 12 of their papers we have counts for
12 papers
PartIR: Composing SPMD Partitioning Strategies for Machine Learning
Sami Alabed, Daniel Belov, Bart Chrzaszcz +14
Training of modern large neural networks (NN) requires a combination of parallelization strategies encompassing data, model, or optimizer sharding. When strategies increase in comp…
Automatic Discovery of Composite SPMD Partitioning Strategies in PartIR
Sami Alabed, Dominik Grewe, Juliana Franco +6
Large neural network models are commonly trained through a combination of advanced parallelism strategies in a single program, multiple data (SPMD) paradigm. For example, training…
Learned Force Fields Are Ready For Ground State Catalyst Discovery
Michael Schaarschmidt, Morgane Riviere, Alex M. Ganose +6
We present evidence that learned density functional theory (``DFT'') force fields are ready for ground state catalyst discovery. Our key finding is that relaxation using forces fro…
Pre-training via Denoising for Molecular Property Prediction
Sheheryar Zaidi, Michael Schaarschmidt, James Martens +6
Many important problems involving molecular property prediction from 3D structures have limited data, posing a generalization challenge for neural networks. In this paper, we descr…
Automap: Towards Ergonomic Automated Parallelism for ML Models
Michael Schaarschmidt, Dominik Grewe, Dimitrios Vytiniotis +8
The rapid rise in demand for training large neural network architectures has brought into focus the need for partitioning strategies, for example by using data, model, or pipeline…
Simple GNN Regularisation for 3D Molecular Property Prediction & Beyond
Jonathan Godwin, Michael Schaarschmidt, Alexander Gaunt +5
In this paper we show that simple noise regularisation can be an effective way to address GNN oversmoothing. First we argue that regularisers addressing oversmoothing should both p…
Learning Index Selection with Structured Action Spaces
Jeremy Welborn, Michael Schaarschmidt, Eiko Yoneki
Configuration spaces for computer systems can be challenging for traditional and automatic tuning strategies. Injecting task-specific knowledge into the tuner for a task may allow…
Wield: Systematic Reinforcement Learning With Progressive Randomization
Michael Schaarschmidt, Kai Fricke, Eiko Yoneki
Reinforcement learning frameworks have introduced abstractions to implement and execute algorithms at scale. They assume standardized simulator interfaces but are not concerned wit…
RLgraph: Modular Computation Graphs for Deep Reinforcement Learning
Michael Schaarschmidt, Sven Mika, Kai Fricke +1
Reinforcement learning (RL) tasks are challenging to implement, execute and test due to algorithmic instability, hyper-parameter sensitivity, and heterogeneous distributed communic…
LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations
Michael Schaarschmidt, Alexander Kuhnle, Ben Ellis +3
Reinforcement learning approaches have long appealed to the data management community due to their ability to learn to control dynamic behavior from raw system performance. Recent…
Tuning the Scheduling of Distributed Stochastic Gradient Descent with Bayesian Optimization
Valentin Dalibard, Michael Schaarschmidt, Eiko Yoneki
We present an optimizer which uses Bayesian optimization to tune the system parameters of distributed stochastic gradient descent (SGD). Given a specific context, our goal is to qu…
Learning Runtime Parameters in Computer Systems with Delayed Experience Injection
Michael Schaarschmidt, Felix Gessert, Valentin Dalibard +1
Learning effective configurations in computer systems without hand-crafting models for every parameter is a long-standing problem. This paper investigates the use of deep reinforce…