Learning Features of Music from Scratch
arXiv:1611.09827
Abstract
This paper introduces a new large-scale music dataset, MusicNet, to serve as a source of supervision and evaluation of machine learning methods for music research. MusicNet consists of hundreds of freely-licensed classical music recordings by 10 composers, written for 11 instruments, together with instrument/note annotations resulting in over 1 million temporal labels on 34 hours of chamber music performances under various studio and microphone conditions. The paper defines a multi-label classification task to predict notes in musical recordings, along with an evaluation protocol, and benchmarks several machine learning architectures for this task: i) learning from spectrogram features; ii) end-to-end learning with a neural net; iii) end-to-end learning with a convolutional neural net. These experiments show that end-to-end models trained for note prediction learn frequency selective filters as a low-level representation of audio.
arXiv comment14 pages; camera-ready version; updated experiments and related works; additional MIR metrics (Appendix C)
References in corpus (2)
Cited by in corpus (28)
- Deep Learning for Audio Signal Processing
- Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
- Generating Multi-Agent Trajectories using Programmatic Weak Supervision
- Onsets and Frames: Dual-Objective Piano Transcription
- Augmentation Methods on Monophonic Audio for Instrument Classification in Polyphonic Music
- Frame-level Instrument Recognition by Timbre and Pitch
- On the Complex Network Structure of Musical Pieces: Analysis of Some Use Cases from Different Music Genres
- Building Blocks for a Complex-Valued Transformer Architecture
- Diffusion-Based Audio Inpainting
- Coupled Recurrent Models for Polyphonic Music Composition
- Sound Event Detection of Weakly Labelled Data with CNN-Transformer and Automatic Threshold Optimization
- A Fully Convolutional Neural Network Approach to End-to-End Speech Enhancement
- nnAudio: An on-the-fly GPU Audio to Spectrogram Conversion Toolbox Using 1D Convolution Neural Networks
- MT3: Multi-Task Multitrack Music Transcription
- ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Data
- Learning Multi-instrument Classification with Partial Labels
- Complex Transformer: A Framework for Modeling Complex-Valued Sequence
- Classical Music Clustering Based on Acoustic Features
- A Hybrid Approach to Audio-to-Score Alignment
- Sequence Prediction using Spectral RNNs
- PerformanceNet: Score-to-Audio Music Generation with Multi-Band Convolutional Residual Network
- Bayesian Sparsification Methods for Deep Complex-valued Networks
- Multitask learning for frame-level instrument recognition
- MULTIMODAL ANALYSIS: Informed content estimation and audio source separation
- A Methodology for Exploring Deep Convolutional Features in Relation to Hand-Crafted Features with an Application to Music Audio Modeling
- Searching Personal Collections
- Multi-layered Cepstrum for Instantaneous Frequency Estimation
- Markov-switching State Space Models for Uncovering Musical Interpretation