66 citations · 84 across the 6 of their papers we have counts for
5 papers · 1 filter
Scalable Pretraining of Large Mixture of Experts Language Models on Aurora Super Computer
Dharma Teja Vooturi, Dhiraj Kalamkar, Dipankar Das +1
Pretraining Large Language Models (LLMs) from scratch requires massive amount of compute. Aurora super computer is an ExaScale machine with 127,488 Intel PVC (Ponte Vechio) GPU til…
AUTOSPARSE: Towards Automated Sparse Training of Deep Neural Networks
Abhisek Kundu, Naveen K. Mellempudi, Dharma Teja Vooturi +2
Sparse training is emerging as a promising avenue for reducing the computational cost of training neural networks. Several recent studies have proposed pruning methods using learna…
Ramanujan Bipartite Graph Products for Efficient Block Sparse Neural Networks
Dharma Teja Vooturi, Girish Varma, Kishore Kothapalli
Sparse neural networks are shown to give accurate predictions competitive to denser versions, while also minimizing the number of arithmetic operations performed. However current h…
A Study of BFLOAT16 for Deep Learning Training
Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi +16
This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for Deep Learning training acros…
Hierarchical Block Sparse Neural Networks
Dharma Teja Vooturi, Dheevatsa Mudigere, Sasikanth Avancha
Sparse deep neural networks(DNNs) are efficient in both memory and compute when compared to dense DNNs. But due to irregularity in computation of sparse DNNs, their efficiencies ar…