Towards pedestrian head tracking: A benchmark dataset and a multi-source data fusion network

Abstract

Pedestrian detection and tracking in crowded video sequences have many applications, including autonomous driving, robot navigation and pedestrian flow analysis. However, detecting and tracking pedestrians in high-density crowds face many challenges, including intra-class occlusions, complex motions, and diverse poses. Although artificial intelligence (AI) models have achieved great progress in head detection, head tracking datasets and methods are extremely lacking. Existing head datasets have limited coverage of complex pedestrian flows and scenes (e.g., pedestrian interactions, occlusions, and object interference). It is of great importance to develop new head tracking datasets and methods. To address these challenges, we present a Chinese Large-scale Cross-scene Pedestrian Head Tracking dataset (Cchead) and a Multi-source Data Fusion Network (MDFN). The dataset has features that are of considerable interest, including 10 diverse scenes of 50,528 frames with about 2,366,249 heads and 2,358 tracks. Our dataset contains diverse pedestrian moving speeds, directions, and complex crowd pedestrian flows with collision avoidance behaviors. Existing state-of-the-art (SOTA) algorithms are tested and compared on the Cchead dataset. MDFN is the first end-to-end convolutional neural network (CNN)-based head detection and tracking network that jointly trains Red, Green, Blue (RGB) frames, pixel-level motion information, depth maps, and density maps in videos. Ablation experiments confirm the significance of multi-source data fusion. Compared with SOTA pedestrian detection and tracking methods, MDFN achieves superior performance across three datasets: Cchead, Restaurant and Crowd of Heads Dataset (CroHD). To promote further development, we share our source code and trained models for global researchers: https://github.com/kailaisun/Cchead.

dataset:https://doi.org/10.34740/kaggle/ds/7494891

References in corpus (15)

• YOLOX: Exceeding YOLO Series in 2021
• FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking
• MOT16: A Benchmark for Multi-Object Tracking
• MOT20: A benchmark for multi object tracking in crowded scenes
• CrowdHuman: A Benchmark for Detecting Human in a Crowd
• 3D Object Detection for Autonomous Driving: A Survey
• BoT-SORT: Robust Associations Multi-Pedestrian Tracking
• A Smart, Efficient, and Reliable Parking Surveillance System with Edge Artificial Intelligence on IoT Devices
• Robotic Vision for Human-Robot Interaction and Collaboration: A Survey and Systematic Review
• Dynamic Traffic Assignment using the Macroscopic Fundamental Diagram: A Review of Vehicular and Pedestrian Flow Models
• CVPR19 Tracking and Detection Challenge: How crowded can it get?
• A Comparison of CNN-based Face and Head Detectors for Real-Time Video Surveillance Applications
• NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimation
• Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking
• MPSN: Motion-aware Pseudo Siamese Network for Indoor Video Head Detection in Buildings