Dropout Feature Ranking for Deep Learning Models

Abstract

Deep neural networks (DNNs) achieve state-of-the-art results in a variety of domains. Unfortunately, DNNs are notorious for their non-interpretability, and thus limit their applicability in hypothesis-driven domains such as biology and healthcare. Moreover, in the resource-constraint setting, it is critical to design tests relying on fewer more informative features leading to high accuracy performance within reasonable budget. We aim to close this gap by proposing a new general feature ranking method for deep learning. We show that our simple yet effective method performs on par or compares favorably to eight strawman, classical and deep-learning feature ranking methods in two simulations and five very different datasets on tasks ranging from classification to regression, in both static and time series scenarios. We also illustrate the use of our method on a drug response dataset and show that it identifies genes relevant to the drug-response.

References in corpus (11)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Learning Important Features Through Propagating Activation Differences
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Grad-CAM: Why did you say that?
• Clinical Intervention Prediction and Understanding using Deep Networks
• Real Time Image Saliency for Black Box Classifiers
• An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis Detection
• A Bayesian encourages dropout
• Dr.VAE: Drug Response Variational Autoencoder

Cited by in corpus (6)

• Recurrent Neural Networks for Time Series Forecasting
• Explainability and Adversarial Robustness for RNNs
• Learning Global Pairwise Interactions with Bayesian Neural Networks
• Deep IDA: A Deep Learning Method for Integrative Discriminant Analysis of Multi-View Data with Feature Ranking -- An Application to COVID-19 severity
• Dropout as a Regularizer of Interaction Effects
• Towards a Better Tradeoff between Effectiveness and Efficiency in Pre-Ranking: A Learnable Feature Selection based Approach