Efficient Crowd Counting via Structured Knowledge Transfer

Abstract

Crowd counting is an application-oriented task and its inference efficiency is crucial for real-world applications. However, most previous works relied on heavy backbone networks and required prohibitive run-time consumption, which would seriously restrict their deployment scopes and cause poor scalability. To liberate these crowd counting models, we propose a novel Structured Knowledge Transfer (SKT) framework, which fully exploits the structured knowledge of a well-trained teacher network to generate a lightweight but still highly effective student network. Specifically, it is integrated with two complementary transfer modules, including an Intra-Layer Pattern Transfer which sequentially distills the knowledge embedded in layer-wise features of the teacher network to guide feature learning of the student network and an Inter-Layer Relation Transfer which densely distills the cross-layer correlation knowledge of the teacher to regularize the student's feature evolutio Consequently, our student network can derive the layer-wise and cross-layer knowledge from the teacher network to learn compact yet effective features. Extensive evaluations on three benchmarks well demonstrate the effectiveness of our SKT for extensive crowd counting models. In particular, only using around $6\%$ of the parameters and computation cost of original models, our distilled VGG-based models obtain at least 6.5$\times$ speed-up on an Nvidia 1080 GPU and even achieve state-of-the-art performance. Our code and models are available at {\url{https://github.com/HCPLab-SYSU/SKT}}.

arXiv commentThis paper has been accepted by ACM MM 2020. Our code and models are available at {\url{https://github.com/HCPLab-SYSU/SKT}}

References in corpus (20)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• FitNets: Hints for Thin Deep Nets
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Compressing Deep Convolutional Networks using Vector Quantization
• Pruning Filters for Efficient ConvNets
• To prune, or not to prune: exploring the efficacy of pruning for model compression
• Crowded Scene Analysis: A Survey
• Convolutional neural networks with low-rank regularization
• Like What You Like: Knowledge Distill via Neuron Selectivity Transfer
• CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes
• Deep Model Compression: Distilling Knowledge from Noisy Teachers
• Crowd Counting and Density Estimation by Trellis Encoder-Decoder Network
• Bayesian Loss for Crowd Count Estimation with Point Supervision
• Deep Learning with Low Precision by Half-wave Gaussian Quantization
• Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds
• Efficient Sparse-Winograd Convolutional Neural Networks
• Crowd Counting with Deep Structured Scale Integration Network
• Crowd Counting using Deep Recurrent Spatial-Aware Network
• Perspective-Guided Convolution Networks for Crowd Counting