Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networks

Abstract

In this work, we research and evaluate end-to-end learning of monocular semantic-metric occupancy grid mapping from weak binocular ground truth. The network learns to predict four classes, as well as a camera to bird's eye view mapping. At the core, it utilizes a variational encoder-decoder network that encodes the front-view visual information of the driving scene and subsequently decodes it into a 2-D top-view Cartesian coordinate system. The evaluations on Cityscapes show that the end-to-end learning of semantic-metric occupancy grids outperforms the deterministic mapping approach with flat-plane assumption by more than 12% mean IoU. Furthermore, we show that the variational sampling with a relatively small embedding vector brings robustness against vehicle dynamic perturbations, and generalizability for unseen KITTI data. Our network achieves real-time inference rates of approx. 35 Hz for an input image with a resolution of 256x512 pixels and an output map with 64x64 occupancy grid cells using a Titan V GPU.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Auto-Encoding Variational Bayes
• Conditional Random Fields as Recurrent Neural Networks
• A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Semantic Segmentation using Adversarial Networks
• Occupancy Grids: A Stochastic Spatial Representation for Active Robot Perception

Cited by in corpus (14)

• SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentation for Accurate Freespace Detection
• Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images
• RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsing
• BEVCar: Camera-Radar Fusion for BEV Map and Object Segmentation
• FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras
• Semantic Foreground Inpainting from Weak Supervision
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study
• Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planning
• Hallucinating Beyond Observation: Learning to Complete with Partial Observation and Unpaired Prior Knowledge
• RendBEV: Semantic Novel View Synthesis for Self-Supervised Bird's Eye View Segmentation
• Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation
• AutoLay: Benchmarking amodal layout estimation for autonomous driving
• Monocular Multi-Layer Layout Estimation for Warehouse Racks
• REFNet++: Multi-Task Efficient Fusion of Camera and Radar Sensor Data in Bird's-Eye Polar View