SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentation for Accurate Freespace Detection

Abstract

Freespace detection is an essential component of visual perception for self-driving cars. The recent efforts made in data-fusion convolutional neural networks (CNNs) have significantly improved semantic driving scene segmentation. Freespace can be hypothesized as a ground plane, on which the points have similar surface normals. Hence, in this paper, we first introduce a novel module, named surface normal estimator (SNE), which can infer surface normal information from dense depth/disparity images with high accuracy and efficiency. Furthermore, we propose a data-fusion CNN architecture, referred to as RoadSeg, which can extract and fuse features from both RGB images and the inferred surface normal information for accurate freespace detection. For research purposes, we publish a large-scale synthetic freespace detection dataset, named Ready-to-Drive (R2D) road dataset, collected under different illumination and weather conditions. The experimental results demonstrate that our proposed SNE module can benefit all the state-of-the-art CNNs for freespace detection, and our SNE-RoadSeg achieves the best overall performance among different datasets.

ECCV 2020

References in corpus (9)

• Rethinking Atrous Convolution for Semantic Image Segmentation
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Pothole Detection Based on Disparity Transformation and Road Surface Modeling
• Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networks
• High-speed Autonomous Drifting with Deep Reinforcement Learning
• Applying Surface Normal Information in Drivable Area and Road Anomaly Detection for Ground Mobile Robots
• DIODE: A Dense Indoor and Outdoor DEpth Dataset
• Three-Filters-to-Normal: An Accurate and Ultrafast Surface Normal Estimator
• Depth-aware CNN for RGB-D Segmentation

Cited by in corpus (24)

• Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation
• Dynamic Fusion Module Evolves Drivable Area and Road Anomaly Detection: A Benchmark and Algorithms
• Semantics for Robotic Mapping, Perception and Interaction: A Survey
• Learning Collision-Free Space Detection from Stereo Images: Homography Matrix Brings Better Data Augmentation
• RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsing
• Multi-Scale Feature Fusion: Learning Better Semantic Segmentation for Road Pothole Detection
• We Learn Better Road Pothole Detection: from Attention Aggregation to Adversarial Domain Adaptation
• VTGNet: A Vision-based Trajectory Generation Network for Autonomous Vehicles in Urban Environments
• Three-Filters-to-Normal: An Accurate and Ultrafast Surface Normal Estimator
• S${}^3$M-Net: Joint Learning of Semantic Segmentation and Stereo Matching for Autonomous Driving
• Pseudo-LiDAR Based Road Detection
• Computer Stereo Vision for Autonomous Driving
• Multi-Modal Multi-Task (3MT) Road Segmentation
• Freespace Optical Flow Modeling for Automated Driving
• Rethinking Road Surface 3D Reconstruction and Pothole Detection: From Perspective Transformation to Disparity Map Segmentation
• DriveGuard: Robustification of Automated Driving Systems with Deep Spatio-Temporal Convolutional Autoencoder
• Agile Reactive Navigation for A Non-Holonomic Mobile Robot Using A Pixel Processor Array
• Establishing Reality-Virtuality Interconnections in Urban Digital Twins for Superior Intelligent Road Inspection and Simulation
• Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation
• S2P2: Self-Supervised Goal-Directed Path Planning Using RGB-D Data for Robotic Wheelchairs
• SCV-Stereo: Learning Stereo Matching from a Sparse Cost Volume
• Co-Teaching: An Ark to Unsupervised Stereo Matching
• Polyline Generative Navigable Space Segmentation for Autonomous Visual Navigation
• NSS-VAEs: Generative Scene Decomposition for Visual Navigable Space Construction