A Review on Deep Learning Techniques Applied to Semantic Segmentation

Abstract

Image semantic segmentation is more and more being of interest for computer vision and machine learning researchers. Many applications on the rise need accurate and efficient segmentation mechanisms: autonomous driving, indoor navigation, and even virtual or augmented reality systems to name a few. This demand coincides with the rise of deep learning approaches in almost every field or application target related to computer vision, including semantic segmentation or scene understanding. This paper provides a review on deep learning methods for semantic segmentation applied to various application areas. Firstly, we describe the terminology of this field as well as mandatory background concepts. Next, the main datasets and challenges are exposed to help researchers decide which are the ones that best suit their needs and their targets. Then, existing methods are reviewed, highlighting their contributions and their significance in the field. Finally, quantitative results are given for the described methods and the datasets in which they were evaluated, following up with a discussion of the results. At last, we point out a set of promising future works and draw our own conclusions about the state of the art of semantic segmentation using deep learning techniques.

arXiv commentSubmitted to TPAMI on Apr. 22, 2017

References in corpus (21)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Semi-Supervised Classification with Graph Convolutional Networks
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Deep Convolutional Networks on Graph-Structured Data
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• ParseNet: Looking Wider to See Better
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Learning Convolutional Neural Networks for Graphs
• Learning to Segment Object Candidates
• Hands Deep in Deep Learning for Hand Pose Estimation
• ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks
• Deep Contrast Learning for Salient Object Detection
• Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts
• A Survey of Semantic Segmentation
• Structured Pruning of Deep Convolutional Neural Networks
• Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge
• Exploiting Local Structures with the Kronecker Layer in Convolutional Networks
• Multi-View Deep Learning for Consistent Semantic Mapping with RGB-D Cameras
• DAG-Recurrent Neural Networks For Scene Labeling

Cited by in corpus (100)

• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
• A Survey on Deep Learning-based Architectures for Semantic Segmentation on 2D images
• Fruit Detection, Segmentation and 3D Visualisation of Environments in Apple Orchards
• Deep-Learning-Based Image Segmentation Integrated with Optical Microscopy for Automatically Searching for Two-Dimensional Materials
• Real-Time Fruit Recognition and Grasping Estimation for Autonomous Apple Harvesting
• DDU-Net: Dual-Decoder-U-Net for Road Extraction Using High-Resolution Remote Sensing Images
• Visual Perception and Modelling in Unstructured Orchard for Apple Harvesting Robots
• Few-Shot Segmentation Propagation with Guided Networks
• BGnet: Accurate and rapid background estimation in single-molecule localization microscopy with deep neural nets
• Building Footprint Generation by IntegratingConvolution Neural Network with Feature PairwiseConditional Random Field (FPCRF)
• A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning
• OFFSEG: A Semantic Segmentation Framework For Off-Road Driving
• Gastric histopathology image segmentation using a hierarchical conditional random field
• Generative Modeling of Turbulence
• MaxiMask and MaxiTrack: two new tools for identifying contaminants in astronomical images using convolutional neural networks
• A Comprehensive Survey of Machine Learning Applied to Radar Signal Processing
• PDNet: Semantic Segmentation integrated with a Primal-Dual Network for Document binarization
• A Comparison and Strategy of Semantic Segmentation on Remote Sensing Images
• Computing Systems for Autonomous Driving: State-of-the-Art and Challenges
• Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review
• Context-Integrated and Feature-Refined Network for Lightweight Object Parsing
• Applying Transfer Learning To Deep Learned Models For EEG Analysis
• Driving Datasets Literature Review
• Shallow2Deep: Indoor Scene Modeling by Single Image Understanding
• Continual Adaptation of Semantic Segmentation using Complementary 2D-3D Data Representations
• LSENet: Location and Seasonality Enhanced Network for Multi-Class Ocean Front Detection
• Multiple Hypothesis Semantic Mapping for Robust Data Association
• Self-supervised Feature Learning for 3D Medical Images by Playing a Rubik's Cube
• Deep Mangoes: from fruit detection to cultivar identification in colour images of mango trees
• Fully Automatic Wound Segmentation with Deep Convolutional Neural Networks
• Semantic segmentation of longitudinal thermal images for identification of hot and cool spots in urban areas
• Juggling With Representations: On the Information Transfer Between Imagery, Point Clouds, and Meshes for Multi-Modal Semantics
• Context-Aware Mixed Reality: A Framework for Ubiquitous Interaction
• DSNet for Real-Time Driving Scene Semantic Segmentation
• Finding Nano-Ötzi: Semi-Supervised Volume Visualization for Cryo-Electron Tomography
• Unbiasing Semantic Segmentation For Robot Perception using Synthetic Data Feature Transfer
• Unsupervised Medical Image Segmentation with Adversarial Networks: From Edge Diagrams to Segmentation Maps
• A Multi-Stage Multi-Task Neural Network for Aerial Scene Interpretation and Geolocalization
• NuClick: A Deep Learning Framework for Interactive Segmentation of Microscopy Images
• A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes
• Mixture Domain Adaptation to Improve Semantic Segmentation in Real-World Surveillance
• Applying Semantic Segmentation to Autonomous Cars in the Snowy Environment
• Kimera-Multi: a System for Distributed Multi-Robot Metric-Semantic Simultaneous Localization and Mapping
• Linear colour segmentation revisited
• TAN: Temporal Affine Network for Real-Time Left Ventricle Anatomical Structure Analysis Based on 2D Ultrasound Videos
• Unsupervised Object Discovery and Segmentation of RGBD-images
• TEDNet: Twin Encoder Decoder Neural Network for 2D Camera and LiDAR Road Detection
• CIRA Guide to Custom Loss Functions for Neural Networks in Environmental Sciences -- Version 1
• A tomographic workflow to enable deep learning for X-ray based foreign object detection
• Condensation-Net: Memory-Efficient Network Architecture with Cross-Channel Pooling Layers and Virtual Feature Maps
• Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data
• Concept-Oriented Deep Learning
• Learn to segment single cells with deep distance estimator and deep cell detector
• Sketch2code: Generating a website from a paper mockup
• On The Coherence of Quantitative Evaluation of Visual Explanations
• Hierarchical semantic segmentation using modular convolutional neural networks
• Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems
• Background Subtraction with Real-time Semantic Segmentation
• Adaptive Ultrasound Beamforming using Deep Learning
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study
• A Classification Refinement Strategy for Semantic Segmentation
• Design and Development of Autonomous Delivery Robot
• A Methodological Review of Visual Road Recognition Procedures for Autonomous Driving Applications
• Deep Dual Pyramid Network for Barcode Segmentation using Barcode-30k Database
• Left Ventricle Segmentation and Quantification from Cardiac Cine MR Images via Multi-task Learning
• TricycleGAN: Unsupervised Image Synthesis and Segmentation Based on Shape Priors
• Computer vision for liquid samples in hospitals and medical labs using hierarchical image segmentation and relations prediction
• Enhanced Self-Perception in Mixed Reality: Egocentric Arm Segmentation and Database with Automatic Labelling
• Auto-Annotation Quality Prediction for Semi-Supervised Learning with Ensembles
• Automatic segmenting teeth in X-ray images: Trends, a novel data set, benchmarking and future perspectives
• Flood Detection On Low Cost Orbital Hardware
• Context-aware Padding for Semantic Segmentation
• Distance to Center of Mass Encoding for Instance Segmentation
• yNet: a multi-input convolutional network for ultra-fast simulation of field evolvement
• CAggNet: Crossing Aggregation Network for Medical Image Segmentation
• Automatic Lumbar Spinal CT Image Segmentation with a Dual Densely Connected U-Net
• Advancing biological super-resolution microscopy through deep learning: a brief review
• Automatic Head Overcoat Thickness Measure with NASNet-Large-Decoder Net
• DFNet: Semantic Segmentation on Panoramic Images with Dynamic Loss Weights and Residual Fusion Block
• Scene Context Based Semantic Segmentation for 3D LiDAR Data in Dynamic Scene
• Focusing on Shadows for Predicting Heightmaps from Single Remotely Sensed RGB Images with Deep Learning
• Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency
• Semantic Segmentation via Highly Fused Convolutional Network with Multiple Soft Cost Functions
• Recurrent convolutional neural networks for mandible segmentation from computed tomography
• Incorporating Human Domain Knowledge in 3D LiDAR-based Semantic Segmentation
• Tissue characterization based on the analysis on i3DUS data for diagnosis support in neurosurgery
• Refined Plane Segmentation for Cuboid-Shaped Objects by Leveraging Edge Detection
• When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey
• Machine Vision for Improved Human-Robot Cooperation in Adverse Underwater Conditions
• A Hybrid Framework for Matching Printing Design Files to Product Photos
• Generic Primitive Detection in Point Clouds Using Novel Minimal Quadric Fits
• 3D Scene Parsing via Class-Wise Adaptation
• Affinity Derivation and Graph Merge for Instance Segmentation
• Accelerated Inference in Markov Random Fields via Smooth Riemannian Optimization
• WeedMap: A large-scale semantic weed mapping framework using aerial multispectral imaging and deep neural network for precision farming
• Iterative multi-path tracking for video and volume segmentation with sparse point supervision
• Presentation Attack Detection for Cadaver Iris
• A Data-driven Prior on Facet Orientation for Semantic Mesh Labeling
• Learning Neural Parsers with Deterministic Differentiable Imitation Learning
• Equivalence of Correlation Filter and Convolution Filter in Visual Tracking