End-to-end learning potentials for structured attribute prediction

Abstract

We present a structured inference approach in deep neural networks for multiple attribute prediction. In attribute prediction, a common approach is to learn independent classifiers on top of a good feature representation. However, such classifiers assume conditional independence on features and do not explicitly consider the dependency between attributes in the inference process. We propose to formulate attribute prediction in terms of marginal inference in the conditional random field. We model potential functions by deep neural networks and apply the sum-product algorithm to solve for the approximate marginal distribution in feed-forward networks. Our message passing layer implements sparse pairwise potentials by a softplus-linear function that is equivalent to a higher-order classifier, and learns all the model parameters by end-to-end back propagation. The experimental results using SUN attributes and CelebA datasets suggest that the structured inference improves the attribute prediction performance, and possibly uncovers the hidden relationship between attributes.

References in corpus (8)

• Deep Residual Learning for Image Recognition
• Sum-Product Networks: A New Deep Architecture
• Joint Object and Part Segmentation using Deep Learned Potentials
• End-to-End Learning for Structured Prediction Energy Networks
• Deep Structured Models For Group Activity Recognition
• Structured Prediction Energy Networks
• Structure Inference Machines: Recurrent Neural Networks for Analyzing Relations in Group Activity Recognition
• End-to-End Training of Hybrid CNN-CRF Models for Stereo