113 citations · 150 across the 18 of their papers we have counts for
3 papers · 1 filter
Modeling Data Reuse in Deep Neural Networks by Taking Data-Types into Cognizance
Nandan Kumar Jha, Sparsh Mittal
In recent years, researchers have focused on reducing the model size and number of computations (measured as "multiply-accumulate" or MAC operations) of DNNs. The energy consumptio…
On the Demystification of Knowledge Distillation: A Residual Network Perspective
Nandan Kumar Jha, Rajat Saini, Sparsh Mittal
Knowledge distillation (KD) is generally considered as a technique for performing model compression and learned-label smoothing. However, in this paper, we study and investigate th…
ULSAM: Ultra-Lightweight Subspace Attention Module for Compact Convolutional Neural Networks
Rajat Saini, Nandan Kumar Jha, Bedanta Das +2
The capability of the self-attention mechanism to model the long-range dependencies has catapulted its deployment in vision models. Unlike convolution operators, self-attention off…