Learning Deconvolution Network for Semantic Segmentation

Abstract

We propose a novel semantic segmentation algorithm by learning a deconvolution network. We learn the network on top of the convolutional layers adopted from VGG 16-layer net. The deconvolution network is composed of deconvolution and unpooling layers, which identify pixel-wise class labels and predict segmentation masks. We apply the trained network to each proposal in an input image, and construct the final semantic segmentation map by combining the results from all proposals in a simple manner. The proposed algorithm mitigates the limitations of the existing methods based on fully convolutional networks by integrating deep deconvolution network and proposal-wise prediction; our segmentation method typically identifies detailed structures and handles objects in multiple scales naturally. Our network demonstrates outstanding performance in PASCAL VOC 2012 dataset, and we achieve the best accuracy (72.5%) among the methods trained with no external data through ensemble with the fully convolutional network.

References in corpus (10)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Going Deeper with Convolutions
• Fully Convolutional Networks for Semantic Segmentation
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Simultaneous Detection and Segmentation
• Feedforward semantic segmentation with zoom-out features

Cited by in corpus (244)

• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• Perceptual Losses for Real-Time Style Transfer and Super-Resolution
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• Fully Convolutional Networks for Semantic Segmentation
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Deep-STORM: super-resolution single-molecule microscopy by deep learning
• Algorithms for Semantic Segmentation of Multispectral Remote Sensing Imagery using Deep Learning
• Stacked Hourglass Networks for Human Pose Estimation
• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey
• Association of genomic subtypes of lower-grade gliomas with shape features automatically extracted by a deep learning algorithm
• Beyond Skip Connections: Top-Down Modulation for Object Detection
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Residual Attention Network for Image Classification
• Pyramid Scene Parsing Network
• Retinal vessel segmentation based on Fully Convolutional Neural Networks
• Modeling and Propagating CNNs in a Tree Structure for Visual Tracking
• Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery
• X-ModalNet: A Semi-Supervised Deep Cross-Modal Network for Classification of Remote Sensing Data
• Rotation equivariant vector field networks
• Pyramid Attention Network for Semantic Segmentation
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
• W-Net: A Deep Model for Fully Unsupervised Image Segmentation
• Learning Depth from Single Images with Deep Neural Network Embedding Focal Length
• Semantic Understanding of Scenes through the ADE20K Dataset
• RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentation
• Dilated Residual Networks
• Deep Residual Learning for Compressed Sensing CT Reconstruction via Persistent Homology Analysis
• Convolutional Neural Fabrics
• CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes
• Convolutional Neural Network-Based Block Up-sampling for Intra Frame Coding
• Multispecies fruit flower detection using a refined semantic segmentation network
• FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos
• CNN-based Segmentation of Medical Imaging Data
• Context Encoding for Semantic Segmentation
• ICNet for Real-Time Semantic Segmentation on High-Resolution Images
• MgNet: A Unified Framework of Multigrid and Convolutional Neural Network
• BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation
• Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection
• ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
• Proposal-free Network for Instance-level Object Segmentation
• PixelNet: Representation of the pixels, by the pixels, and for the pixels
• Object Detection with Deep Learning: A Review
• Efficient piecewise training of deep structured models for semantic segmentation
• Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction
• Stacked Deconvolutional Network for Semantic Segmentation
• Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization
• Hierarchical Convolutional-Deconvolutional Neural Networks for Automatic Liver and Tumor Segmentation
• Deep generative-contrastive networks for facial expression recognition
• Cost-Effective Active Learning for Melanoma Segmentation
• RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation
• PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing
• Learning to Refine Object Segments
• CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos
• PixelNet: Towards a General Pixel-level Architecture
• Unsupervised Histopathology Image Synthesis
• TricorNet: A Hybrid Temporal Convolutional and Recurrent Network for Video Action Segmentation
• Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network
• Label Refinement Network for Coarse-to-Fine Semantic Segmentation
• A Closed-form Solution to Photorealistic Image Stylization
• Deep Spatial Feature Reconstruction for Partial Person Re-identification: Alignment-Free Approach
• Looking at Outfit to Parse Clothing
• What's the Point: Semantic Segmentation with Point Supervision
• A Hybrid Convolutional Variational Autoencoder for Text Generation
• SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks
• Addressing Overfitting on Pointcloud Classification using Atrous XCRF
• Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation
• Combining the Best of Convolutional Layers and Recurrent Layers: A Hybrid Network for Semantic Segmentation
• A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA
• Image Companding and Inverse Halftoning using Deep Convolutional Neural Networks
• Two-Phase Learning for Weakly Supervised Object Localization
• Combining Background Subtraction Algorithms with Convolutional Neural Network
• Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
• Deep Active Contours
• Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform
• Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features
• Predicting dark matter halo formation in N-body simulations with deep regression networks
• TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes
• Stacked U-Nets: A No-Frills Approach to Natural Image Segmentation
• Face Parsing via a Fully-Convolutional Continuous CRF Neural Network
• Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation
• Pixel Deconvolutional Networks
• DeepUNet: A Deep Fully Convolutional Network for Pixel-level Sea-Land Segmentation
• Exploiting saliency for object segmentation from image level labels
• Weakly Supervised Semantic Segmentation using Web-Crawled Videos
• Monocular Depth Estimation using Multi-Scale Continuous CRFs as Sequential Deep Networks
• Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation
• Segmentation from Natural Language Expressions
• ZM-Net: Real-time Zero-shot Image Manipulation Network
• High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis
• Joint Sequence Learning and Cross-Modality Convolution for 3D Biomedical Segmentation
• Deep Learning For Face Recognition: A Critical Analysis
• Zoom Out-and-In Network with Recursive Training for Object Proposal
• Block-Matching Convolutional Neural Network for Image Denoising
• No More Discrimination: Cross City Adaptation of Road Scene Segmenters
• Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs
• Learning Uncertain Convolutional Features for Accurate Saliency Detection
• Crowd counting via scale-adaptive convolutional neural network
• Improving Fully Convolution Network for Semantic Segmentation
• DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation
• Real-Time Facial Segmentation and Performance Capture from RGB Input
• Efficient Yet Deep Convolutional Neural Networks for Semantic Segmentation
• An Adversarial Regularisation for Semi-Supervised Training of Structured Output Neural Networks
• Multi-View Deep Learning for Consistent Semantic Mapping with RGB-D Cameras
• Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries
• Deep artifact learning for compressed sensing and parallel MRI
• Effective Use of Synthetic Data for Urban Scene Semantic Segmentation
• Synthetic contrast enhancement in cardiac CT with Deep Learning
• Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes
• Unified Perceptual Parsing for Scene Understanding
• Dense Transformer Networks
• First Person Action-Object Detection with EgoNet
• Discriminatively Boosted Image Clustering with Fully Convolutional Auto-Encoders
• Structured Knowledge Distillation for Dense Prediction
• A New Convolutional Network-in-Network Structure and Its Applications in Skin Detection, Semantic Segmentation, and Artifact Reduction
• Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics
• Revisiting Perspective Information for Efficient Crowd Counting
• Iris and periocular recognition in arabian race horses using deep convolutional neural networks
• Deep GrabCut for Object Selection
• Deconvolution-Based Global Decoding for Neural Machine Translation
• 3D Human Pose Estimation from a Single Image via Distance Matrix Regression
• Convolutional Random Walk Networks for Semantic Image Segmentation
• Adaptive Affinity Fields for Semantic Segmentation
• Zoom Better to See Clearer: Human and Object Parsing with Hierarchical Auto-Zoom Net
• Deconvolutional Feature Stacking for Weakly-Supervised Semantic Segmentation
• CSVideoNet: A Real-time End-to-end Learning Framework for High-frame-rate Video Compressive Sensing
• Understanding Traffic Density from Large-Scale Web Camera Data
• SAN: Scale-Aware Network for Semantic Segmentation of High-Resolution Aerial Images
• Exploring Context with Deep Structured models for Semantic Segmentation
• MSR: Multi-Scale Shape Regression for Scene Text Detection
• Control Distance IoU and Control Distance IoU Loss Function for Better Bounding Box Regression
• Learning Deep Representations for Semantic Image Parsing: a Comprehensive Overview
• Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks
• Deep Heterogeneous Feature Fusion for Template-Based Face Recognition
• A feasibility study for predicting optimal radiation therapy dose distributions of prostate cancer patients from patient anatomy using deep learning
• Recent Advances in the Applications of Convolutional Neural Networks to Medical Image Contour Detection
• Decoupled Spatial Neural Attention for Weakly Supervised Semantic Segmentation
• Developing a seismic pattern interpretation network (SpiNet) for automated seismic interpretation
• Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network
• X-GANs: Image Reconstruction Made Easy for Extreme Cases
• Gaussian Processes Semantic Map Representation
• Efficient Road Lane Marking Detection with Deep Learning
• Towards Automatic Image Editing: Learning to See another You
• Tree-structured Kronecker Convolutional Network for Semantic Segmentation
• SketchParse : Towards Rich Descriptions for Poorly Drawn Sketches using Multi-Task Hierarchical Deep Networks
• Weakly Supervised Object Localization Using Things and Stuff Transfer
• Generative Model with Coordinate Metric Learning for Object Recognition Based on 3D Models
• Real-time Human Pose Estimation from Video with Convolutional Neural Networks
• ZipNet-GAN: Inferring Fine-grained Mobile Traffic Patterns via a Generative Adversarial Neural Network
• Surveillance Video Parsing with Single Frame Supervision
• Dense Recurrent Neural Networks for Scene Labeling
• Saliency Detection by Forward and Backward Cues in Deep-CNNs
• Dynamic-structured Semantic Propagation Network
• Learn to segment single cells with deep distance estimator and deep cell detector
• Towards Lifelong Self-Supervision: A Deep Learning Direction for Robotics
• SOGNet: Scene Overlap Graph Network for Panoptic Segmentation
• One-to-Many Network for Visually Pleasing Compression Artifacts Reduction
• Weak-shot Semantic Segmentation by Transferring Semantic Affinity and Boundary
• Dynamic Neural Representational Decoders for High-Resolution Semantic Segmentation
• A Multiscale Patch Based Convolutional Network for Brain Tumor Segmentation
• Zoom Out-and-In Network with Map Attention Decision for Region Proposal and Object Detection
• Bottom-Up and Top-Down Reasoning with Hierarchical Rectified Gaussians
• Accurate Automatic Segmentation of Amygdala Subnuclei and Modeling of Uncertainty via Bayesian Fully Convolutional Neural Network
• Fast and Accurate Semantic Mapping through Geometric-based Incremental Segmentation
• A Network Structure to Explicitly Reduce Confusion Errors in Semantic Segmentation
• Demand Forecasting from Spatiotemporal Data with Graph Networks and Temporal-Guided Embedding
• Scene Labeling using Gated Recurrent Units with Explicit Long Range Conditioning
• Triply Supervised Decoder Networks for Joint Detection and Segmentation
• Neuron-level Selective Context Aggregation for Scene Segmentation
• A Deep Learning Approach Based on Graphs to Detect Plantation Lines
• Identifying Most Walkable Direction for Navigation in an Outdoor Environment
• ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation
• Generalized Deep Image to Image Regression
• EfficientHRNet: Efficient Scaling for Lightweight High-Resolution Multi-Person Pose Estimation
• Learning Deep Representations Using Convolutional Auto-encoders with Symmetric Skip Connections
• SeGAN: Segmenting and Generating the Invisible
• CNN based texture synthesize with Semantic segment
• Low-Latency Video Semantic Segmentation
• Butterfly Effect: Bidirectional Control of Classification Performance by Small Additive Perturbation
• Deep Dual Pyramid Network for Barcode Segmentation using Barcode-30k Database
• Towards Clinical Diagnosis: Automated Stroke Lesion Segmentation on Multimodal MR Image Using Convolutional Neural Network
• Learning Deep Representations for Scene Labeling with Semantic Context Guided Supervision
• 3DContextNet: K-d Tree Guided Hierarchical Learning of Point Clouds Using Local and Global Contextual Cues
• Landmark Detection in Low Resolution Faces with Semi-Supervised Learning
• A Compact DNN: Approaching GoogLeNet-Level Accuracy of Classification and Domain Adaptation
• Multilevel Context Representation for Improving Object Recognition
• Deeply-Supervised CNN for Prostate Segmentation
• Fast Object Localization Using a CNN Feature Map Based Multi-Scale Search
• Rethinking Convolutional Semantic Segmentation Learning
• Chinese/English mixed Character Segmentation as Semantic Segmentation
• Deconvolution-and-convolution Networks
• VidFace: A Full-Transformer Solver for Video FaceHallucination with Unaligned Tiny Snapshots
• Efficient and Invariant Convolutional Neural Networks for Dense Prediction
• BENCHIP: Benchmarking Intelligence Processors
• DSNet: Deep and Shallow Feature Learning for Efficient Visual Tracking
• One-Shot Segmentation in Clutter
• STD2P: RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling
• Hierarchical Back Projection Network for Image Super-Resolution
• Convolutional Neural Networks to Enhance Coded Speech
• Nazr-CNN: Fine-Grained Classification of UAV Imagery for Damage Assessment
• Detect, Replace, Refine: Deep Structured Prediction For Pixel Wise Labeling
• Disentangling Pose from Appearance in Monochrome Hand Images
• DeMeshNet: Blind Face Inpainting for Deep MeshFace Verification
• More Information Supervised Probabilistic Deep Face Embedding Learning
• CRAM: Clued Recurrent Attention Model
• Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks
• Correlating Edge, Pose with Parsing
• Scene Parsing via Dense Recurrent Neural Networks with Attentional Selection
• CardiacNET: Segmentation of Left Atrium and Proximal Pulmonary Veins from MRI Using Multi-View CNN
• ESFNet: Efficient Network for Building Extraction from High-Resolution Aerial Images
• Enhancement Mask for Hippocampus Detection and Segmentation
• Fed-EINI: An Efficient and Interpretable Inference Framework for Decision Tree Ensembles in Federated Learning
• A Multi-Modal States based Vehicle Descriptor and Dilated Convolutional Social Pooling for Vehicle Trajectory Prediction
• Learning to Segment Corneal Tissue Interfaces in OCT Images
• A Domain Agnostic Normalization Layer for Unsupervised Adversarial Domain Adaptation
• Out of the Black Box: Properties of deep neural networks and their applications
• PARN: Pyramidal Affine Regression Networks for Dense Semantic Correspondence
• SynNet: Structure-Preserving Fully Convolutional Networks for Medical Image Synthesis
• Excavate Condition-invariant Space by Intrinsic Encoder
• Semantic Segmentation via Highly Fused Convolutional Network with Multiple Soft Cost Functions
• FastVentricle: Cardiac Segmentation with ENet
• Efficient Convolutional Neural Network with Binary Quantization Layer
• Deep Neural Networks Regularization for Structured Output Prediction
• A Holistic Approach for Data-Driven Object Cutout
• Learning to segment with image-level supervision
• Region-based semantic segmentation with end-to-end training
• Orthogonally Regularized Deep Networks For Image Super-resolution
• Left ventricle segmentation By modelling uncertainty in prediction of deep convolutional neural networks and adaptive thresholding inference
• FHEDN: A based on context modeling Feature Hierarchy Encoder-Decoder Network for face detection
• Multi-Scale Spatially-Asymmetric Recalibration for Image Classification
• Attention to Refine through Multi-Scales for Semantic Segmentation
• DeepWheat: Estimating Phenotypic Traits from Crop Images with Deep Learning
• Inner Space Preserving Generative Pose Machine
• A Brief Survey and an Application of Semantic Image Segmentation for Autonomous Driving
• Finding Correspondences for Optical Flow and Disparity Estimations using a Sub-pixel Convolution-based Encoder-Decoder Network
• Deep Semantics-Aware Photo Adjustment
• Heterogeneous Grid Convolution for Adaptive, Efficient, and Controllable Computation
• Split-Merge Pooling
• MERIT: Tensor Transform for Memory-Efficient Vision Processing on Parallel Architectures
• Investigations on the inference optimization techniques and their impact on multiple hardware platforms for Semantic Segmentation
• Superpixel-based Semantic Segmentation Trained by Statistical Process Control
• A CNN Cascade for Landmark Guided Semantic Part Segmentation