Deep convolutional filter banks for texture recognition and segmentation

Abstract

Research in texture recognition often concentrates on the problem of material recognition in uncluttered conditions, an assumption rarely met by applications. In this work we conduct a first study of material and describable texture at- tributes recognition in clutter, using a new dataset derived from the OpenSurface texture repository. Motivated by the challenge posed by this problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN substantially improves the state-of-the-art in texture, mate- rial and scene recognition. Our approach achieves 82.3% accuracy on Flickr material dataset and 81.1% accuracy on MIT indoor scenes, providing absolute gains of more than 10% over existing approaches. D-CNN easily trans- fers across domains without requiring feature adaptation as for methods that build on the fully-connected layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale information and describe regions of arbitrary shapes and sizes. Our approach is particularly suited at lo- calizing stuff categories and obtains state-of-the-art re- sults on MSRC segmentation dataset, as well as promising results on recognizing materials and surface attributes in clutter on the OpenSurfaces dataset.

Accepted to CVPR15

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• CNN: Single-label to Multi-label
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• Simultaneous Detection and Segmentation

Cited by in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Texture Synthesis Using Convolutional Neural Networks
• Deep Image: Scaling up Image Recognition
• Hybrid CNN and Dictionary-Based Models for Scene Recognition and Domain Adaptation
• Universal representations:The missing link between faces, text, planktons, and cat breeds
• Deep Spatial Pyramid: The Devil is Once Again in the Details
• Accelerating Very Deep Convolutional Networks for Classification and Detection
• Geometric Operator Convolutional Neural Network
• Convolutional Neural Network on Three Orthogonal Planes for Dynamic Texture Classification
• Visual Recognition Using Directional Distribution Distance
• Backpropagation Training for Fisher Vectors within Neural Networks
• Compositional Model based Fisher Vector Coding for Image Classification
• Scalable Nonlinear Embeddings for Semantic Category-based Image Retrieval
• Leveraging Tacit Information Embedded in CNN Layers for Visual Tracking