Material Recognition in the Wild with the Materials in Context Database

Abstract

Recognizing materials in real-world images is a challenging task. Real-world materials have rich surface texture, geometry, lighting conditions, and clutter, which combine to make the problem particularly difficult. In this paper, we introduce a new, large-scale, open dataset of materials in the wild, the Materials in Context Database (MINC), and combine this dataset with deep learning to achieve material recognition and segmentation of images in the wild. MINC is an order of magnitude larger than previous material databases, while being more diverse and well-sampled across its 23 categories. Using MINC, we train convolutional neural networks (CNNs) for two tasks: classifying materials from patches, and simultaneous material recognition and segmentation in full images. For patch-based classification on MINC we found that the best performing CNN architectures can achieve 85.2% mean class accuracy. We convert these trained CNN classifiers into an efficient fully convolutional framework combined with a fully connected conditional random field (CRF) to predict the material at every pixel in an image, achieving 73.1% mean class accuracy. Our experiments demonstrate that having a large, well-sampled dataset such as MINC is crucial for real-world material recognition and segmentation.

CVPR 2015. Sean Bell and Paul Upchurch contributed equally

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Going Deeper with Convolutions
• Rich feature hierarchies for accurate object detection and semantic segmentation

Cited by in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Evaluating color texture descriptors under large variations of controlled lighting conditions
• Learning Visual Clothing Style with Heterogeneous Dyadic Co-occurrences
• Deep Learning for Tactile Understanding From Visual and Haptic Data
• Training object class detectors with click supervision
• Learning Material-Aware Local Descriptors for 3D Shapes
• Deep Outdoor Illumination Estimation
• Deep Texture Manifold for Ground Terrain Recognition
• Deep TEN: Texture Encoding Network
• A 4D Light-Field Dataset and CNN Architectures for Material Recognition
• Statistically Motivated Second Order Pooling