Crowd Counting by Adapting Convolutional Neural Networks with Side Information

Abstract

Computer vision tasks often have side information available that is helpful to solve the task. For example, for crowd counting, the camera perspective (e.g., camera angle and height) gives a clue about the appearance and scale of people in the scene. While side information has been shown to be useful for counting systems using traditional hand-crafted features, it has not been fully utilized in counting systems based on deep learning. In order to incorporate the available side information, we propose an adaptive convolutional neural network (ACNN), where the convolutional filter weights adapt to the current scene context via the side information. In particular, we model the filter weights as a low-dimensional manifold, parametrized by the side information, within the high-dimensional space of filter weights. With the help of side information and adaptive weights, the ACNN can disentangle the variations related to the side information, and extract discriminative features related to the current context. Since existing crowd counting datasets do not contain ground-truth side information, we collect a new dataset with the ground-truth camera angle and height as the side information. On experiments in crowd counting, the ACNN improves counting accuracy compared to a plain CNN with a similar number of parameters. We also apply ACNN to image deconvolution to show its potential effectiveness on other computer vision applications.

8 pages

Cited by in corpus (6)

• Exploiting Unlabeled Data in CNNs by Self-supervised Learning to Rank
• Fast Video Crowd Counting with a Temporal Aware Network
• Leveraging Unlabeled Data for Crowd Counting by Learning to Rank
• User Constrained Thumbnail Generation using Adaptive Convolutions
• Multi-channel Deep Supervision for Crowd Counting
• Functional Neural Networks for Parametric Image Restoration Problems