113 citations · 121 across the 3 of their papers we have counts for
Showing cs.CVShow all
2 papers · 1 filter
cs.CV2020★ 2 cited
On the Demystification of Knowledge Distillation: A Residual Network Perspective
Nandan Kumar Jha, Rajat Saini, Sparsh Mittal
Knowledge distillation (KD) is generally considered as a technique for performing model compression and learned-label smoothing. However, in this paper, we study and investigate th…
cs.CV2020★ 113 cited
ULSAM: Ultra-Lightweight Subspace Attention Module for Compact Convolutional Neural Networks
Rajat Saini, Nandan Kumar Jha, Bedanta Das +2
The capability of the self-attention mechanism to model the long-range dependencies has catapulted its deployment in vision models. Unlike convolution operators, self-attention off…