PixelNet: Representation of the pixels, by the pixels, and for the pixels

Abstract

We explore design principles for general pixel-level prediction problems, from low-level edge detection to mid-level surface normal estimation to high-level semantic segmentation. Convolutional predictors, such as the fully-convolutional network (FCN), have achieved remarkable success by exploiting the spatial redundancy of neighboring pixels through convolutional processing. Though computationally efficient, we point out that such approaches are not statistically efficient during learning precisely because spatial redundancy limits the information learned from neighboring pixels. We demonstrate that stratified sampling of pixels allows one to (1) add diversity during batch updates, speeding up learning; (2) explore complex nonlinear predictors, improving accuracy; and (3) efficiently train state-of-the-art models tabula rasa (i.e., "from scratch") for diverse pixel-labeling tasks. Our single architecture produces state-of-the-art results for semantic segmentation on PASCAL-Context dataset, surface normal estimation on NYUDv2 depth dataset, and edge detection on BSDS.

Project Page: http://www.cs.cmu.edu/~aayushb/pixelNet/. arXiv admin note: substantial text overlap with arXiv:1609.06694

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Learning Deconvolution Network for Semantic Segmentation
• FlowNet: Learning Optical Flow with Convolutional Networks
• An Implementation of Faster RCNN with Study for Region Sampling
• Pixel-wise Deep Learning for Contour Detection
• Analyzing the Performance of Multilayer Neural Networks for Object Recognition
• Feedforward semantic segmentation with zoom-out features
• Designing Deep Networks for Surface Normal Estimation
• Oriented Edge Forests for Boundary Detection

Cited by in corpus (13)

• GeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation
• Dense Recurrent Neural Networks for Scene Labeling
• PixelNN: Example-based Image Synthesis
• Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery
• DivergentNets: Medical Image Segmentation by Network Ensemble
• Video Representation Learning and Latent Concept Mining for Large-scale Multi-label Video Classification
• Streaming Self-Training via Domain-Agnostic Unlabeled Images
• Explainable Semantic Mapping for First Responders
• Improving task-specific representation via 1M unlabelled images without any extra knowledge
• Weakly-Supervised Cell Tracking via Backward-and-Forward Propagation
• Superpixel-based Semantic Segmentation Trained by Statistical Process Control
• Glioma Prognosis: Segmentation of the Tumor and Survival Prediction using Shape, Geometric and Clinical Information
• Sparse2Dense: From direct sparse odometry to dense 3D reconstruction