Object Detectors Emerge in Deep Scene CNNs

Abstract

With the success of new computational architectures for visual processing, such as convolutional neural networks (CNN) and access to image databases with millions of labeled examples (e.g., ImageNet, Places), the state of the art in computer vision is advancing rapidly. One important factor for continued progress is to understand the representations that are learned by the inner layers of these deep architectures. Here we show that object detectors emerge from training CNNs to perform scene classification. As scenes are composed of objects, the CNN for scene classification automatically discovers meaningful objects detectors, representative of the learned scene categories. With object detectors emerging as a result of learning to recognize scenes, our work demonstrates that the same network can perform both scene recognition and object localization in a single forward-pass, without ever having been explicitly taught the notion of objects.

arXiv comment12 pages, ICLR 2015 conference paper

References in corpus (2)

• Learning Generative Models with Visual Attention
• Self-taught Object Localization with Deep Networks

Cited by in corpus (138)

• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• ParseNet: Looking Wider to See Better
• Anabranch Network for Camouflaged Object Segmentation
• Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation
• Learning to Generate Reviews and Discovering Sentiment
• Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision
• Pyramid Scene Parsing Network
• What makes ImageNet good for transfer learning?
• Semantic Labeling in Very High Resolution Images via a Self-Cascaded Convolutional Neural Network
• Concept Whitening for Interpretable Image Recognition
• Do Convolutional Neural Networks Learn Class Hierarchy?
• Looking Outside the Window: Wide-Context Transformer for the Semantic Segmentation of High-Resolution Remote Sensing Images
• Restricted Deformable Convolution based Road Scene Semantic Segmentation Using Surround View Cameras
• Semantic-Aware Scene Recognition
• Interpreting and Improving Adversarial Robustness of Deep Neural Networks with Neuron Sensitivity
• Look Wider to Match Image Patches with Convolutional Neural Networks
• Zero-Shot Detection
• What Do We Understand About Convolutional Networks?
• Weakly Supervised Medical Diagnosis and Localization from Multiple Resolutions
• Locally-Supervised Deep Hybrid Model for Scene Recognition
• Direct-Manipulation Visualization of Deep Networks
• Diving Deep into Sentiment: Understanding Fine-tuned CNNs for Visual Sentiment Prediction
• Tell Me Where to Look: Guided Attention Inference Network
• Visual and Semantic Knowledge Transfer for Large Scale Semi-supervised Object Detection
• See, Hear, and Read: Deep Aligned Representations
• Scene Recognition with Prototype-agnostic Scene Layout
• A Survey of Deep Learning for Scientific Discovery
• A Taxonomy and Library for Visualizing Learned Features in Convolutional Neural Networks
• Range Loss for Deep Face Recognition with Long-tail
• How convolutional neural network see the world - A survey of convolutional neural network visualization methods
• Conceptual Spaces for Cognitive Architectures: A Lingua Franca for Different Levels of Representation
• On Interpretability of Artificial Neural Networks: A Survey
• Prospects for Theranostics in Neurosurgical Imaging: Empowering Confocal Laser Endomicroscopy Diagnostics via Deep Learning
• Attention-guided Chained Context Aggregation for Semantic Segmentation
• Incorporating Network Built-in Priors in Weakly-supervised Semantic Segmentation
• It Takes Two to Tango: Towards Theory of AI's Mind
• Object Discovery From a Single Unlabeled Image by Mining Frequent Itemset With Multi-scale Features
• Deep Convolutional Features for Image Based Retrieval and Scene Categorization
• Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation
• Visual pathways from the perspective of cost functions and multi-task deep neural networks
• Exploring Categorical Regularization for Domain Adaptive Object Detection
• Recognizing Characters in Art History Using Deep Learning
• A New Method to Visualize Deep Neural Networks
• Deep Contextual Attention for Human-Object Interaction Detection
• Learning to count with deep object features
• Soft Proposal Networks for Weakly Supervised Object Localization
• Now You See Me (CME): Concept-based Model Extraction
• Semantic Instance Annotation of Street Scenes by 3D to 2D Label Transfer
• Understanding and Predicting the Memorability of Outdoor Natural Scenes
• Emergent Properties of Foveated Perceptual Systems
• 3D Bounding Box Estimation Using Deep Learning and Geometry
• Global Aggregation then Local Distribution for Scene Parsing
• Quantifying Legibility of Indoor Spaces Using Deep Convolutional Neural Networks: Case Studies in Train Stations
• Feature Removal Is a Unifying Principle for Model Explanation Methods
• Evaluating the visualization of what a Deep Neural Network has learned
• Structure-Aware Network for Lane Marker Extraction with Dynamic Vision Sensor
• Machine learning surrogates for efficient hydrologic modeling: Insights from stochastic simulations of managed aquifer recharge
• Attention to Head Locations for Crowd Counting
• Convolutional neural network based decoders for surface codes
• Why my photos look sideways or upside down? Detecting Canonical Orientation of Images using Convolutional Neural Networks
• Towards Counterfactual and Contrastive Explainability and Transparency of DCNN Image Classifiers
• Funnel Activation for Visual Recognition
• Visual Relationship Detection using Scene Graphs: A Survey
• Lung Cancer Screening Using Adaptive Memory-Augmented Recurrent Networks
• Learning Shape Priors for Single-View 3D Completion and Reconstruction
• Adjustable Bounded Rectifiers: Towards Deep Binary Representations
• Distilling Localization for Self-Supervised Representation Learning
• Multi-Object Classification and Unsupervised Scene Understanding Using Deep Learning Features and Latent Tree Probabilistic Models
• Expert identification of visual primitives used by CNNs during mammogram classification
• How To Extract Fashion Trends From Social Media? A Robust Object Detector With Support For Unsupervised Learning
• The Role of Typicality in Object Classification: Improving The Generalization Capacity of Convolutional Neural Networks
• Understanding Individual Decisions of CNNs via Contrastive Backpropagation
• Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images
• Targeted Nonlinear Adversarial Perturbations in Images and Videos
• Adversarial Machine Learning Phases of Matter
• Disentangled Image Matting
• Deep Neural Networks for Choice Analysis: A Statistical Learning Theory Perspective
• A Study on Multimodal and Interactive Explanations for Visual Question Answering
• Interpretable Convolutional Neural Networks via Feedforward Design
• MineGAN: effective knowledge transfer from GANs to target domains with few images
• Efficient Training of Very Deep Neural Networks for Supervised Hashing
• Semantics for Global and Local Interpretation of Deep Neural Networks
• Explainable Abstract Trains Dataset
• ArbiText: Arbitrary-Oriented Text Detection in Unconstrained Scene
• Self-Supervised Visual Place Recognition Learning in Mobile Robots
• From Heatmaps to Structural Explanations of Image Classifiers
• Visual Concept Recognition and Localization via Iterative Introspection
• Human perception in computer vision
• Learning Interpretable Concept Groups in CNNs
• Deep Neural Networks for Choice Analysis: Extracting Complete Economic Information for Interpretation
• Graph Saliency Maps through Spectral Convolutional Networks: Application to Sex Classification with Brain Connectivity
• Deep Multi-Modal Image Correspondence Learning
• Discover and Learn New Objects from Documentaries
• Quantifying Learnability and Describability of Visual Concepts Emerging in Representation Learning
• Mask Encoding for Single Shot Instance Segmentation
• Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training
• Fast Crack Detection Using Convolutional Neural Network
• A Selective Survey on Versatile Knowledge Distillation Paradigm for Neural Network Models
• Learning Action Maps of Large Environments via First-Person Vision
• Deep Learning for Robust Motion Segmentation with Non-Static Cameras
• SurfConv: Bridging 3D and 2D Convolution for RGBD Images
• Deep Visual City Recognition Visualization
• Learning Channel Inter-dependencies at Multiple Scales on Dense Networks for Face Recognition
• Improved Deep Learning of Object Category using Pose Information
• Weakly Supervised PET Tumor Detection Using Class Response
• Diverse Sampling for Self-Supervised Learning of Semantic Segmentation
• Deep Learning in the Era of Edge Computing: Challenges and Opportunities
• DeepEverest: Accelerating Declarative Top-K Queries for Deep Neural Network Interpretation
• Model Interpretability and Rationale Extraction by Input Mask Optimization
• Interpretable Neural Network Decoupling
• On the Evolution of Neuron Communities in a Deep Learning Architecture
• Finding Discriminative Filters for Specific Degradations in Blind Super-Resolution
• Mining Interpretable AOG Representations from Convolutional Networks via Active Question Answering
• Improving Spatiotemporal Self-Supervision by Deep Reinforcement Learning
• MFPP: Morphological Fragmental Perturbation Pyramid for Black-Box Model Explanations
• Deep neural networks can be improved using human-derived contextual expectations
• Deep Neural Networks for Choice Analysis: Architectural Design with Alternative-Specific Utility Functions
• Multi-Level Contextual Network for Biomedical Image Segmentation
• Optimising the Input Image to Improve Visual Relationship Detection
• Understanding Character Recognition using Visual Explanations Derived from the Human Visual System and Deep Networks
• Demystifying Brain Tumour Segmentation Networks: Interpretability and Uncertainty Analysis
• Integrated Grad-CAM: Sensitivity-Aware Visual Explanation of Deep Convolutional Networks via Integrated Gradient-Based Scoring
• Splenomegaly Segmentation on Multi-modal MRI using Deep Convolutional Networks
• Semantic Network Interpretation
• Low-Cost Transfer Learning of Face Tasks
• Image Score: How to Select Useful Samples
• Exponential Discriminative Metric Embedding in Deep Learning
• Fine-Grained Categorization via CNN-Based Automatic Extraction and Integration of Object-Level and Part-Level Features
• Verifiability and Predictability: Interpreting Utilities of Network Architectures for Point Cloud Processing
• AMC-Loss: Angular Margin Contrastive Loss for Improved Explainability in Image Classification
• Interpretable Attention Guided Network for Fine-grained Visual Classification
• Capturing Localized Image Artifacts through a CNN-based Hyper-image Representation
• Ada-SISE: Adaptive Semantic Input Sampling for Efficient Explanation of Convolutional Neural Networks
• Interpretability in Safety-Critical FinancialTrading Systems
• Understanding Convolutional Neural Networks with A Mathematical Model
• Mapping the Internet: Modelling Entity Interactions in Complex Heterogeneous Networks
• Toward a Realistic Benchmark for Out-of-Distribution Detection
• Attending Category Disentangled Global Context for Image Classification