148 citations · 557 across the 22 of their papers we have counts for
25 papers
Scalable Pretraining of Large Mixture of Experts Language Models on Aurora Super Computer
Dharma Teja Vooturi, Dhiraj Kalamkar, Dipankar Das +1
Pretraining Large Language Models (LLMs) from scratch requires massive amount of compute. Aurora super computer is an ExaScale machine with 127,488 Intel PVC (Ponte Vechio) GPU til…
Multi-modal encoder-decoder neural network for forecasting solar wind speed at L1
Dattaraj B. Dhuri, Shravan M. Hanasoge, Harsh Joon +3
The solar wind, accelerated within the solar corona, sculpts the heliosphere and continuously interacts with planetary atmospheres. On Earth, high-speed solar-wind streams may lead…
Generative Active Learning for the Search of Small-molecule Protein Binders
Maksym Korablyov, Cheng-Hao Liu, Moksh Jain +31
Despite substantial progress in machine learning for scientific discovery in recent years, truly de novo design of small molecules which exhibit a property of interest remains a si…
AUTOSPARSE: Towards Automated Sparse Training of Deep Neural Networks
Abhisek Kundu, Naveen K. Mellempudi, Dharma Teja Vooturi +2
Sparse training is emerging as a promising avenue for reducing the computational cost of training neural networks. Several recent studies have proposed pruning methods using learna…
Measuring frequency and period separations in red-giant stars using machine learning
Siddharth Dhanpal, Othman Benomar, Shravan Hanasoge +4
Asteroseismology is used to infer the interior physics of stars. The \textit{Kepler} and TESS space missions have provided a vast data set of red-giant light curves, which may be u…
Efficient and Generic 1D Dilated Convolution Layer for Deep Learning
Narendra Chaudhary, Sanchit Misra, Dhiraj Kalamkar +5
Convolutional neural networks (CNNs) have found many applications in tasks involving two-dimensional (2D) data, such as image classification and image processing. Therefore, 2D con…
AI Powered Compiler Techniques for DL Code Optimization
Sanket Tavarageri, Gagandeep Goyal, Sasikanth Avancha +2
Creating high performance implementations of deep learning primitives on CPUs is a challenging task. Multiple considerations including multi-level cache hierarchy, and wide SIMD un…
GNNerator: A Hardware/Software Framework for Accelerating Graph Neural Networks
Jacob R. Stevens, Dipankar Das, Sasikanth Avancha +2
Graph Neural Networks (GNNs) use a fully-connected layer to extract features from the nodes of a graph and aggregate these features using message passing between nodes, combining t…
MADRaS : Multi Agent Driving Simulator
Anirban Santara, Sohan Rudra, Sree Aditya Buridi +4
In this work, we present MADRaS, an open-source multi-agent driving simulator for use in the design and evaluation of motion planning algorithms for autonomous driving. MADRaS prov…
PolyDL: Polyhedral Optimizations for Creation of High Performance DL primitives
Sanket Tavarageri, Alexander Heinecke, Sasikanth Avancha +3
Deep Neural Networks (DNNs) have revolutionized many aspects of our lives. The use of DNNs is becoming ubiquitous including in softwares for image recognition, speech recognition,…
PolyScientist: Automatic Loop Transformations Combined with Microkernels for Optimization of Deep Learning Primitives
Sanket Tavarageri, Alexander Heinecke, Sasikanth Avancha +3
At the heart of deep learning training and inferencing are computationally intensive primitives such as convolutions which form the building blocks of deep neural networks. Researc…
SEERL: Sample Efficient Ensemble Reinforcement Learning
Rohan Saphal, Balaraman Ravindran, Dheevatsa Mudigere +2
Ensemble learning is a very prevalent method employed in machine learning. The relative success of ensemble methods is attributed to their ability to tackle a wide range of instanc…
K-TanH: Efficient TanH For Deep Learning
Abhisek Kundu, Alex Heinecke, Dhiraj Kalamkar +7
We propose K-TanH, a novel, highly accurate, hardware efficient approximation of popular activation function TanH for Deep Learning. K-TanH consists of parameterized low-precision…
High Performance Scalable FPGA Accelerator for Deep Neural Networks
Sudarshan Srinivasan, Pradeep Janedula, Saurabh Dhoble +7
Low-precision is the first order knob for achieving higher Artificial Intelligence Operations (AI-TOPS). However the algorithmic space for sub-8-bit precision compute is diverse, w…
A Study of BFLOAT16 for Deep Learning Training
Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi +16
This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for Deep Learning training acros…
Automatic Model Parallelism for Deep Neural Networks with Compiler and Hardware Support
Sanket Tavarageri, Srinivas Sridharan, Bharat Kaul
The deep neural networks (DNNs) have been enormously successful in tasks that were hitherto in the human-only realm such as image recognition, and language translation. Owing to th…
Mixed Precision Training With 8-bit Floating Point
Naveen Mellempudi, Sudarshan Srinivasan, Dipankar Das +1
Reduced precision computation for deep neural networks is one of the key areas addressing the widening compute gap driven by an exponential growth in model size. In recent years, d…
Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifiers
Apoorv Vyas, Nataraj Jammalamadaka, Xia Zhu +3
As deep learning methods form a critical part in commercially important applications such as autonomous driving and medical diagnostics, it is important to reliably detect out-of-d…
Mixed Precision Training of Convolutional Neural Networks using Integer Operations
Dipankar Das, Naveen Mellempudi, Dheevatsa Mudigere +14
The state-of-the-art (SOTA) for mixed precision training is dominated by variants of low precision floating point operations, and in particular, FP16 accumulating into FP32 Micikev…
On Scale-out Deep Learning Training for Cloud and HPC
Srinivas Sridharan, Karthikeyan Vaidyanathan, Dhiraj Kalamkar +8
The exponential growth in use of large deep neural networks has accelerated the need for training these deep neural networks in hours or even minutes. This can only be achieved thr…
RAIL: Risk-Averse Imitation Learning
Anirban Santara, Abhishek Naik, Balaraman Ravindran +4
Imitation learning algorithms learn viable policies by imitating an expert's behavior when reward signals are not available. Generative Adversarial Imitation Learning (GAIL) is a s…
Ternary Residual Networks
Abhisek Kundu, Kunal Banerjee, Naveen Mellempudi +4
Sub-8-bit representation of DNNs incur some discernible loss of accuracy despite rigorous (re)training at low-precision. Such loss of accuracy essentially makes them equivalent to…
Ternary Neural Networks with Fine-Grained Quantization
Naveen Mellempudi, Abhisek Kundu, Dheevatsa Mudigere +3
We propose a novel fine-grained quantization (FGQ) method to ternarize pre-trained full precision models, while also constraining activations to 8 and 4-bits. Using this method, we…
Mixed Low-precision Deep Learning Inference using Dynamic Fixed Point
Naveen Mellempudi, Abhisek Kundu, Dipankar Das +2
We propose a cluster-based quantization method to convert pre-trained full precision weights into ternary weights with minimal impact on the accuracy. In addition, we also constrai…