113 citations · 121 across the 3 of their papers we have counts for
3 papers
On the Demystification of Knowledge Distillation: A Residual Network Perspective
Nandan Kumar Jha, Rajat Saini, Sparsh Mittal
Knowledge distillation (KD) is generally considered as a technique for performing model compression and learned-label smoothing. However, in this paper, we study and investigate th…
ULSAM: Ultra-Lightweight Subspace Attention Module for Compact Convolutional Neural Networks
Rajat Saini, Nandan Kumar Jha, Bedanta Das +2
The capability of the self-attention mechanism to model the long-range dependencies has catapulted its deployment in vision models. Unlike convolution operators, self-attention off…
E2GC: Energy-efficient Group Convolution in Deep Neural Networks
Nandan Kumar Jha, Rajat Saini, Subhrajit Nag +1
The number of groups () in group convolution (GConv) is selected to boost the predictive performance of deep neural networks (DNNs) in a compute and parameter efficient manner.…