Learning Deep Structured Models

Abstract

Many problems in real-world applications involve predicting several random variables which are statistically related. Markov random fields (MRFs) are a great mathematical tool to encode such relationships. The goal of this paper is to combine MRFs with deep learning algorithms to estimate complex representations while taking into account the dependencies between the output random variables. Towards this goal, we propose a training algorithm that is able to learn structured models jointly with deep features that form the MRF potentials. Our approach is efficient as it blends learning and inference and makes use of GPU acceleration. We demonstrate the effectiveness of our algorithm in the tasks of predicting words from noisy images, as well as multi-class classification of Flickr photographs. We show that joint learning of the deep features and the MRF parameters results in significant performance gains.

11 pages including reference

References in corpus (7)

• Natural Language Processing (almost) from Scratch
• Conditional Random Fields as Recurrent Neural Networks
• Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
• Fully Connected Deep Structured Networks
• Tightening LP Relaxations for MAP using Message Passing
• Convergent message passing algorithms - a unifying view
• Efficient Structured Prediction with Latent Variables for General Graphical Models

Cited by in corpus (19)

• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Conditional Random Fields as Recurrent Neural Networks
• Discriminative Embeddings of Latent Variable Models for Structured Data
• Multi-Label Classification Neural Networks with Hard Logical Constraints
• Pixel-Adaptive Convolutional Neural Networks
• Adaptive Graphical Model Network for 2D Handpose Estimation
• Relational Neural Machines
• Frank-Wolfe Network: An Interpretable Deep Structure for Non-Sparse Coding
• Multi-Object Classification and Unsupervised Scene Understanding Using Deep Learning Features and Latent Tree Probabilistic Models
• LP-SparseMAP: Differentiable Relaxed Optimization for Sparse Structured Prediction
• Structured Output Learning with Conditional Generative Flows
• Learning Discriminators as Energy Networks in Adversarial Learning
• Taxonomy of Dual Block-Coordinate Ascent Methods for Discrete Energy Minimization
• End-to-End Learned Random Walker for Seeded Image Segmentation
• Scaling Matters in Deep Structured-Prediction Models
• End-to-end Training of CNN-CRF via Differentiable Dual-Decomposition
• MPLP++: Fast, Parallel Dual Block-Coordinate Ascent for Dense Graphical Models
• Learning Propagation for Arbitrarily-structured Data
• Adversarial Structure Matching for Structured Prediction Tasks