Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

Abstract

Most state-of-the-art techniques for multi-class image segmentation and labeling use conditional random fields defined over pixels or image regions. While region-level models often feature dense pairwise connectivity, pixel-level models are considerably larger and have only permitted sparse graph structures. In this paper, we consider fully connected CRF models defined on the complete set of pixels in an image. The resulting graphs have billions of edges, making traditional inference algorithms impractical. Our main contribution is a highly efficient approximate inference algorithm for fully connected CRF models in which the pairwise edge potentials are defined by a linear combination of Gaussian kernels. Our experiments demonstrate that dense connectivity at the pixel level substantially improves segmentation and labeling accuracy.

NIPS 2011

Cited by in corpus (85)

• Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields
• Learning Deconvolution Network for Semantic Segmentation
• Visual Saliency Detection Based on Multiscale Deep CNN Features
• DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy
• Fully Connected Deep Structured Networks
• Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects
• W-Net: A Deep Model for Fully Unsupervised Image Segmentation
• Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks
• Playing for Data: Ground Truth from Computer Games
• Gated-SCNN: Gated Shape CNNs for Semantic Segmentation
• PixelNet: Representation of the pixels, by the pixels, and for the pixels
• Automatic Liver Lesion Detection using Cascaded Deep Residual Networks
• RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation
• PixelNet: Towards a General Pixel-level Architecture
• segDeepM: Exploiting Segmentation and Context in Deep Neural Networks for Object Detection
• SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks
• Incorporating Network Built-in Priors in Weakly-supervised Semantic Segmentation
• MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features
• Interactive Video Object Segmentation in the Wild
• A Context-aware Delayed Agglomeration Framework for Electron Microscopy Segmentation
• Face Parsing via a Fully-Convolutional Continuous CRF Neural Network
• Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation
• Learning Affinity via Spatial Propagation Networks
• A Survey on Deep Learning Architectures for Image-based Depth Reconstruction
• Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation
• InstanceCut: from Edges to Instances with MultiCut
• Holistic, Instance-Level Human Parsing
• Improving Fully Convolution Network for Semantic Segmentation
• Semantic Motion Segmentation Using Dense CRF Formulation
• Effective Use of Dilated Convolutions for Segmenting Small Object Instances in Remote Sensing Imagery
• LabelBank: Revisiting Global Perspectives for Semantic Segmentation
• Learning the Matching Function
• Temporal Convolutional Networks: A Unified Approach to Action Segmentation
• Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes
• Deep GrabCut for Object Selection
• A Deep Primal-Dual Network for Guided Depth Super-Resolution
• Cystoid macular edema segmentation of Optical Coherence Tomography images using fully convolutional neural networks and fully connected CRFs
• FoveaNet: Perspective-aware Urban Scene Parsing
• Adversarial Deep Structured Nets for Mass Segmentation from Mammograms
• Self-explanatory Deep Salient Object Detection
• A Gap-Based Framework for Chinese Word Segmentation via Very Deep Convolutional Networks
• Dense Recurrent Neural Networks for Scene Labeling
• Error Correction for Dense Semantic Image Labeling
• Global Hypothesis Generation for 6D Object Pose Estimation
• Learning Normalized Inputs for Iterative Estimation in Medical Image Segmentation
• A Regularized Convolutional Neural Network for Semantic Image Segmentation
• Beyond Forward Shortcuts: Fully Convolutional Master-Slave Networks (MSNets) with Backward Skip Connections for Semantic Segmentation
• Can Ground Truth Label Propagation from Video help Semantic Segmentation?
• Neuron-level Selective Context Aggregation for Scene Segmentation
• Deep Stereo Matching with Dense CRF Priors
• Scene Labeling using Gated Recurrent Units with Explicit Long Range Conditioning
• Mixed context networks for semantic segmentation
• A Classification Refinement Strategy for Semantic Segmentation
• Conditional Random Field and Deep Feature Learning for Hyperspectral Image Segmentation
• Perturb-and-MPM: Quantifying Segmentation Uncertainty in Dense Multi-Label CRFs
• An Artificial Intelligence-Based System for Nutrient Intake Assessment of Hospitalised Patients
• Rethinking Convolutional Semantic Segmentation Learning
• Belief Propagation in Conditional RBMs for Structured Prediction
• Detect, Replace, Refine: Deep Structured Prediction For Pixel Wise Labeling
• Learning Dynamic Hierarchical Models for Anytime Scene Labeling
• Diverse Sampling for Self-Supervised Learning of Semantic Segmentation
• Kinematic-Layout-aware Random Forests for Depth-based Action Recognition
• Progressively Diffused Networks for Semantic Image Segmentation
• Beyond Pixels: A Comprehensive Survey from Bottom-up to Semantic Image Segmentation and Cosegmentation
• Sequential Labeling with online Deep Learning
• Person re-identification via efficient inference in fully connected CRF
• A Tiered Move-making Algorithm for General Non-submodular Pairwise Energies
• Deep, Dense, and Low-Rank Gaussian Conditional Random Fields
• Deep Neural Networks In Fully Connected CRF For Image Labeling With Social Network Metadata
• Weakly Supervised Image Annotation and Segmentation with Objects and Attributes
• Supervised Hashing based on Energy Minimization
• Per-Pixel Feedback for improving Semantic Segmentation
• Closed-Loop Adaptation for Weakly-Supervised Semantic Segmentation
• Worst-case Optimal Submodular Extensions for Marginal Estimation
• Gland Instance Segmentation by Deep Multichannel Side Supervision
• Region-based semantic segmentation with end-to-end training
• Saliency Detection via Combining Region-Level and Pixel-Level Predictions with CNNs
• Efficient Continuous Relaxations for Dense CRF
• A Holistic Approach for Data-Driven Object Cutout
• Forming A Random Field via Stochastic Cliques: From Random Graphs to Fully Connected Random Fields
• Multi-Scale Convolutions for Learning Context Aware Feature Representations
• Low-level Vision by Consensus in a Spatial Hierarchy of Regions
• Context Tricks for Cheap Semantic Segmentation
• Dense Semantic Correspondence where Every Pixel is a Classifier
• Video Semantic Object Segmentation by Self-Adaptation of DCNN