Illuminating Pedestrians via Simultaneous Detection & Segmentation

Abstract

Pedestrian detection is a critical problem in computer vision with significant impact on safety in urban autonomous driving. In this work, we explore how semantic segmentation can be used to boost pedestrian detection accuracy while having little to no impact on network efficiency. We propose a segmentation infusion network to enable joint supervision on semantic segmentation and pedestrian detection. When placed properly, the additional supervision helps guide features in shared layers to become more sophisticated and helpful for the downstream pedestrian detector. Using this approach, we find weakly annotated boxes to be sufficient for considerable performance gains. We provide an in-depth analysis to demonstrate how shared layers are shaped by the segmentation supervision. In doing so, we show that the resulting feature maps become more semantically meaningful and robust to shape and occlusion. Overall, our simultaneous detection and segmentation framework achieves a considerable gain over the state-of-the-art on the Caltech pedestrian dataset, competitive performance on KITTI, and executes 2x faster than competitive methods.

References in corpus (15)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Residual Learning for Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Going Deeper with Convolutions
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Simultaneous Detection and Segmentation
• Scale-aware Fast R-CNN for Pedestrian Detection
• How Far are We from Solving Pedestrian Detection?
• Instance-aware Semantic Segmentation via Multi-task Network Cascades
• Filtered Channel Features for Pedestrian Detection
• Learning Complexity-Aware Cascades for Deep Pedestrian Detection
• Deep Learning for Semantic Part Segmentation with High-Level Guidance
• Fused DNN: A deep neural network fusion approach to fast and robust pedestrian detection
• Is Faster R-CNN Doing Well for Pedestrian Detection?

Cited by in corpus (16)

• Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation
• The Adaptability and Challenges of Autonomous Vehicles to Pedestrians in Urban China
• Hybrid Channel Based Pedestrian Detection
• Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection
• Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems
• Unsupervised Hard Example Mining from Videos for Improved Object Detection
• GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection
• PSC-Net: Learning Part Spatial Co-occurrence for Occluded Pedestrian Detection
• Triply Supervised Decoder Networks for Joint Detection and Segmentation
• Pedestrian Detection with Autoregressive Network Phases
• Resisting Crowd Occlusion and Hard Negatives for Pedestrian Detection in the Wild
• Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blocks
• Generating Easy-to-Understand Referring Expressions for Target Identifications
• Recurrent Flow-Guided Semantic Forecasting
• Partially fake it till you make it: mixing real and fake thermal images for improved object detection
• Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstruction