Restricted Deformable Convolution based Road Scene Semantic Segmentation Using Surround View Cameras

Abstract

Understanding the surrounding environment of the vehicle is still one of the challenges for autonomous driving. This paper addresses 360-degree road scene semantic segmentation using surround view cameras, which are widely equipped in existing production cars. First, in order to address large distortion problem in the fisheye images, Restricted Deformable Convolution (RDC) is proposed for semantic segmentation, which can effectively model geometric transformations by learning the shapes of convolutional filters conditioned on the input feature map. Second, in order to obtain a large-scale training set of surround view images, a novel method called zoom augmentation is proposed to transform conventional images to fisheye images. Finally, an RDC based semantic segmentation model is built; the model is trained for real-world surround view images through a multi-task learning architecture by combining real-world images with transformed images. Experiments demonstrate the effectiveness of the RDC to handle images with large distortions, and that the proposed approach shows a good performance using surround view cameras with the help of the transformed images.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• Object Detectors Emerge in Deep Scene CNNs
• Revisiting Batch Normalization For Practical Domain Adaptation
• Incorporating Network Built-in Priors in Weakly-supervised Semantic Segmentation
• DecomposeMe: Simplifying ConvNets for End-to-End Learning
• Dense Transformer Networks

Cited by in corpus (18)

• Advancements in Point Cloud Data Augmentation for Deep Learning: A Survey
• RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic Segmentation
• Weighted Intersection over Union (wIoU) for Evaluating Image Segmentation
• A Comprehensive Review of Modern Object Segmentation Approaches
• OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression
• Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges
• Adaptable Deformable Convolutions for Semantic Segmentation of Fisheye Images in Autonomous Driving Systems
• DS-PASS: Detail-Sensitive Panoramic Annular Semantic Segmentation through SwaftNet for Surrounding Sensing
• DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange
• STRIDE: Street View-based Environmental Feature Detection and Pedestrian Collision Prediction
• Energy-Efficient Accelerator Design for Deformable Convolution Networks
• ODE-CNN: Omnidirectional Depth Extension Networks
• Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation
• Universal Semantic Segmentation for Fisheye Urban Driving Images
• Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation
• Capturing Omni-Range Context for Omnidirectional Segmentation
• Semantic Segmentation of Panoramic Images Using a Synthetic Dataset
• Efficient Semantic Segmentation for Visual Bird's-eye View Interpretation