347 citations · 392 across the 7 of their papers we have counts for
6 papers · 1 filter
PopSparse: Accelerated block sparse matrix multiplication on IPU
Zhiyi Li, Douglas Orr, Valeriu Ohan +5
Reducing the computational cost of running large scale neural networks using sparsity has attracted great attention in the deep learning community. While much success has been achi…
GPS++: Reviving the Art of Message Passing for Molecular Property Prediction
Dominic Masters, Josef Dean, Kerstin Klaser +9
We present GPS++, a hybrid Message Passing Neural Network / Graph Transformer model for molecular property prediction. Our model integrates a well-tuned local message passing compo…
8-bit Numerical Formats for Deep Neural Networks
Badreddine Noune, Philip Jones, Daniel Justus +2
Given the current trend of increasing size and complexity of machine learning architectures, it has become of critical importance to identify new approaches to improve the computat…
Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence
Antoine Labatie, Dominic Masters, Zach Eaton-Rosen +1
We investigate the reasons for the performance degradation incurred with batch-independent normalization. We find that the prototypical techniques of layer normalization and instan…
Making EfficientNet More Efficient: Exploring Batch-Independent Normalization, Group Convolutions and Reduced Resolution Training
Dominic Masters, Antoine Labatie, Zach Eaton-Rosen +1
Much recent research has been dedicated to improving the efficiency of training and inference for image classification. This effort has commonly focused on explicitly improving the…
Revisiting Small Batch Training for Deep Neural Networks
Dominic Masters, Carlo Luschi
Modern deep neural network training is typically based on mini-batch stochastic gradient optimization. While the use of large mini-batches increases the available computational par…