148 citations · 428 across the 18 of their papers we have counts for
6 papers · 1 filter
Scalable Pretraining of Large Mixture of Experts Language Models on Aurora Super Computer
Dharma Teja Vooturi, Dhiraj Kalamkar, Dipankar Das +1
Pretraining Large Language Models (LLMs) from scratch requires massive amount of compute. Aurora super computer is an ExaScale machine with 127,488 Intel PVC (Ponte Vechio) GPU til…
Efficient and Generic 1D Dilated Convolution Layer for Deep Learning
Narendra Chaudhary, Sanchit Misra, Dhiraj Kalamkar +5
Convolutional neural networks (CNNs) have found many applications in tasks involving two-dimensional (2D) data, such as image classification and image processing. Therefore, 2D con…
DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networks
Vasimuddin Md, Sanchit Misra, Guixiang Ma +6
Full-batch training on Graph Neural Networks (GNN) to learn the structure of large graphs is a critical problem that needs to scale to hundreds of compute nodes to be feasible. It…
K-TanH: Efficient TanH For Deep Learning
Abhisek Kundu, Alex Heinecke, Dhiraj Kalamkar +7
We propose K-TanH, a novel, highly accurate, hardware efficient approximation of popular activation function TanH for Deep Learning. K-TanH consists of parameterized low-precision…
High-Performance Deep Learning via a Single Building Block
Evangelos Georganas, Kunal Banerjee, Dhiraj Kalamkar +6
Deep learning (DL) is one of the most prominent branches of machine learning. Due to the immense computational cost of DL workloads, industry and academia have developed DL librari…
A Study of BFLOAT16 for Deep Learning Training
Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi +16
This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for Deep Learning training acros…