66 citations · 137 across the 11 of their papers we have counts for
6 papers · 1 filter
FP8 Formats for Deep Learning
Paulius Micikevicius, Dusan Stosic, Neil Burgess +12
FP8 is a natural progression for accelerating deep learning training inference beyond the 16-bit formats common in modern processors. In this paper we propose an 8-bit floating poi…
Efficient and Generic 1D Dilated Convolution Layer for Deep Learning
Narendra Chaudhary, Sanchit Misra, Dhiraj Kalamkar +5
Convolutional neural networks (CNNs) have found many applications in tasks involving two-dimensional (2D) data, such as image classification and image processing. Therefore, 2D con…
DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networks
Vasimuddin Md, Sanchit Misra, Guixiang Ma +6
Full-batch training on Graph Neural Networks (GNN) to learn the structure of large graphs is a critical problem that needs to scale to hundreds of compute nodes to be feasible. It…
Training Neural Machine Translation (NMT) Models using Tensor Train Decomposition on TensorFlow (T3F)
Amelia Drew, Alexander Heinecke
We implement a Tensor Train layer in the TensorFlow Neural Machine Translation (NMT) model using the t3f library. We perform training runs on the IWSLT English-Vietnamese '15 and W…
High-Performance Deep Learning via a Single Building Block
Evangelos Georganas, Kunal Banerjee, Dhiraj Kalamkar +6
Deep learning (DL) is one of the most prominent branches of machine learning. Due to the immense computational cost of DL workloads, industry and academia have developed DL librari…
A Study of BFLOAT16 for Deep Learning Training
Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi +16
This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for Deep Learning training acros…