Motion-guided Non-local Spatial-Temporal Network for Video Crowd Counting
arXiv:2104.13946
Abstract
We study video crowd counting, which is to estimate the number of objects (people in this paper) in all the frames of a video sequence. Previous work on crowd counting is mostly on still images. There has been little work on how to properly extract and take advantage of the spatial-temporal correlation between neighboring frames in both short and long ranges to achieve high estimation accuracy for a video sequence. In this work, we propose Monet, a novel and highly accurate motion-guided non-local spatial-temporal network for video crowd counting. Monet first takes people flow (motion information) as guidance to coarsely segment the regions of pixels where a person may be. Given these regions, Monet then uses a non-local spatial-temporal network to extract spatial-temporally both short and long-range contextual information. The whole network is finally trained end-to-end with a fused loss to generate a high-quality density map. Noting the scarcity and low quality (in terms of resolution and scene diversity) of the publicly available video crowd datasets, we have collected and built a large-scale video crowd counting datasets, VidCrowd, to contribute to the community. VidCrowd contains 9,000 frames of high resolution (2560 x 1440), with 1,150,239 head annotations captured in different scenes, crowd density and lighting in two cities. We have conducted extensive experiments on the challenging VideoCrowd and two public video crowd counting datasets: UCSD and Mall. Our approach achieves substantially better performance in terms of MAE and MSE as compared with other state-of-the-art approaches.
References in corpus (7)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Distribution Matching for Crowd Counting
- Crowd Counting by Adaptively Fusing Predictions from an Image Pyramid
- Spatiotemporal Dilated Convolution with Uncertain Matching for Video-based Crowd Estimation
- Learning from Synthetic Data for Crowd Counting in the Wild
- Enhanced 3D convolutional networks for crowd counting
- Automatic microscopic cell counting by use of unsupervised adversarial domain adaptation and supervised density regression