571 citations · 1.7k across the 33 of their papers we have counts for
5 papers · 1 filter
Efficient Distributed Hessian Free Algorithm for Large-scale Empirical Risk Minimization via Accumulating Sample Strategy
Majid Jahani, Xi He, Chenxin Ma +4
In this paper, we propose a Distributed Accumulated Newton Conjugate gradiEnt (DANCE) method in which sample size is gradually increasing to quickly obtain a solution whose empiric…
Hierarchical Block Sparse Neural Networks
Dharma Teja Vooturi, Dheevatsa Mudigere, Sasikanth Avancha
Sparse deep neural networks(DNNs) are efficient in both memory and compute when compared to dense DNNs. But due to irregularity in computation of sparse DNNs, their efficiencies ar…
Mixed Precision Training of Convolutional Neural Networks using Integer Operations
Dipankar Das, Naveen Mellempudi, Dheevatsa Mudigere +14
The state-of-the-art (SOTA) for mixed precision training is dominated by variants of low precision floating point operations, and in particular, FP16 accumulating into FP32 Micikev…
A Progressive Batching L-BFGS Method for Machine Learning
Raghu Bollapragada, Dheevatsa Mudigere, Jorge Nocedal +2
The standard L-BFGS method relies on gradient approximations that are not dominated by noise, so that search directions are descent directions, the line search is reliable, and qua…
On Scale-out Deep Learning Training for Cloud and HPC
Srinivas Sridharan, Karthikeyan Vaidyanathan, Dhiraj Kalamkar +8
The exponential growth in use of large deep neural networks has accelerated the need for training these deep neural networks in hours or even minutes. This can only be achieved thr…