Contextual Classification Using Self-Supervised Auxiliary Models for Deep Neural Networks

Abstract

Classification problems solved with deep neural networks (DNNs) typically rely on a closed world paradigm, and optimize over a single objective (e.g., minimization of the cross-entropy loss). This setup dismisses all kinds of supporting signals that can be used to reinforce the existence or absence of a particular pattern. The increasing need for models that are interpretable by design makes the inclusion of said contextual signals a crucial necessity. To this end, we introduce the notion of Self-Supervised Autogenous Learning (SSAL) models. A SSAL objective is realized through one or more additional targets that are derived from the original supervised classification task, following architectural principles found in multi-task learning. SSAL branches impose low-level priors into the optimization process (e.g., grouping). The ability of using SSAL branches during inference, allow models to converge faster, focusing on a richer set of class-relevant features. We show that SSAL models consistently outperform the state-of-the-art while also providing structured predictions that are more interpretable.

arXiv commentAccepted for publication at the International Conference of Pattern Recognition (ICPR) 2020

References in corpus (10)

• Distilling the Knowledge in a Neural Network
• Intriguing properties of neural networks
• Towards A Rigorous Science of Interpretable Machine Learning
• Squeeze-and-Excitation Networks
• Wide Residual Networks
• Deeply-Supervised Nets
• Towards Open World Recognition
• One Model To Learn Them All
• Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification
• Complement Objective Training

Cited by in corpus (1)

• Breast Cancer Histopathology Image based Gene Expression Prediction using Spatial Transcriptomics data and Deep Learning