A Methodology for Exploring Deep Convolutional Features in Relation to Hand-Crafted Features with an Application to Music Audio Modeling

Abstract

Understanding the features learned by deep models is important from a model trust perspective, especially as deep systems are deployed in the real world. Most recent approaches for deep feature understanding or model explanation focus on highlighting input data features that are relevant for classification decisions. In this work, we instead take the perspective of relating deep features to well-studied, hand-crafted features that are meaningful for the application of interest. We propose a methodology and set of systematic experiments for exploring deep features in this setting, where input feature importance approaches for deep feature understanding do not apply. Our experiments focus on understanding which hand-crafted and deep features are useful for the classification task of interest, how robust these features are for related tasks and how similar the deep features are to the meaningful hand-crafted features. Our proposed method is general to many application areas and we demonstrate its utility on orchestral music audio data.

arXiv commentCode available at https://github.com/aky4wn/convolutions-for-music-audio

References in corpus (13)

• WaveNet: A Generative Model for Raw Audio
• Axiomatic Attribution for Deep Networks
• Learning Important Features Through Propagating Activation Differences
• Deep Image Prior
• Deep Scattering Spectrum
• Sanity Checks for Saliency Maps
• Similarity of Neural Network Representations Revisited
• A Benchmark for Interpretability Methods in Deep Neural Networks
• Convergent Learning: Do different neural networks learn the same representations?
• Learning Features of Music from Scratch
• Rethinking CNN Models for Audio Classification
• Joint Time-Frequency Scattering
• Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation