Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription

Abstract

We investigate the problem of modeling symbolic sequences of polyphonic music in a completely general piano-roll representation. We introduce a probabilistic model based on distribution estimators conditioned on a recurrent neural network that is able to discover temporal dependencies in high-dimensional sequences. Our approach outperforms many traditional models of polyphonic music on a variety of realistic datasets. We show how our musical language model can serve as a symbolic prior to improve the accuracy of polyphonic transcription.

Appears in Proceedings of the 29th International Conference on Machine Learning (ICML 2012)

References in corpus (1)

• Conditional Restricted Boltzmann Machines for Structured Output Prediction

Cited by in corpus (133)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• LSTM: A Search Space Odyssey
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• On the difficulty of training Recurrent Neural Networks
• A Recurrent Latent Variable Model for Sequential Data
• Recent Advances in Recurrent Neural Networks
• MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation
• Deep Structured Energy Based Models for Anomaly Detection
• How to Construct Deep Recurrent Neural Networks
• Learning Stochastic Recurrent Networks
• Dynamical Variational Autoencoders: A Comprehensive Review
• Variational Recurrent Auto-Encoders
• DeepBach: a Steerable Model for Bach Chorales Generation
• Learning Contextual Dependencies with Convolutional Hierarchical Recurrent Neural Networks
• Compressing Recurrent Neural Network with Tensor Train
• Representation Learning: A Review and New Perspectives
• R-Transformer: Recurrent Neural Network Enhanced Transformer
• A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions
• MorpheuS: generating structured music with constrained patterns and tension
• Counterpoint by Convolution
• Music transcription modelling and composition using deep learning
• DeepJ: Style-Specific Music Generation
• On the Potential of Simple Framewise Approaches to Piano Transcription
• Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization
• MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer
• Deep Temporal Sigmoid Belief Networks for Sequence Modeling
• GACELA -- A generative adversarial context encoder for long audio inpainting
• POP909: A Pop-song Dataset for Music Arrangement Generation
• Imposing higher-level Structure in Polyphonic Music Generation using Convolutional Restricted Boltzmann Machines and Constraints
• Time-series modeling with undecimated fully convolutional neural networks
• LSTM Based Music Generation System
• Neural Adaptive Sequential Monte Carlo
• A Unit Selection Methodology for Music Generation Using Deep Neural Networks
• MMM : Exploring Conditional Multi-Track Music Generation with the Transformer
• Nonparametric Bayesian Factor Analysis for Dynamic Count Matrices
• Learning a Latent Space of Multitrack Measures
• Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation
• Z-Forcing: Training Stochastic Recurrent Networks
• State Space LSTM Models with Particle MCMC Inference
• Deep Learning for Music
• Reweighted Wake-Sleep
• Recurrent Gaussian Processes
• Generating synthetic mobility data for a realistic population with RNNs to improve utility and privacy
• Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control
• MusPy: A Toolkit for Symbolic Music Generation
• An energy-based generative sequence model for testing sensory theories of Western harmony
• Interactive Music Generation with Positional Constraints using Anticipation-RNNs
• PIANOTREE VAE: Structured Representation Learning for Polyphonic Music
• Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data
• Bach in 2014: Music Composition with Recurrent Neural Network
• Towards Score Following in Sheet Music Images
• Transformation Autoregressive Networks
• Energy-based Models for Video Anomaly Detection
• The Statistical Recurrent Unit
• LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks
• Modeling Musical Context with Word2vec
• Melody Generation using an Interactive Evolutionary Algorithm
• Knowledge Transfer Pre-training
• Controllable deep melody generation via hierarchical music structure representation
• Getting Closer to the Essence of Music: The Con Espressione Manifesto
• Learning to Adapt by Minimizing Discrepancy
• From Artificial Neural Networks to Deep Learning for Music Generation -- History, Concepts and Trends
• A Classifying Variational Autoencoder with Application to Polyphonic Music Generation
• A Hierarchical Recurrent Neural Network for Symbolic Melody Generation
• Recurrent Neural Networks with Stochastic Layers for Acoustic Novelty Detection
• Kronecker Recurrent Units
• Implementing a Bayes Filter in a Neural Circuit: The Case of Unknown Stimulus Dynamics
• Adaptive Learning Method of Recurrent Temporal Deep Belief Network to Analyze Time Series Data
• NONOTO: A Model-agnostic Web Interface for Interactive Music Composition by Inpainting
• InfoCatVAE: Representation Learning with Categorical Variational Autoencoders
• A database linking piano and orchestral MIDI scores with application to automatic projective orchestration
• Learning Style-Aware Symbolic Music Representations by Adversarial Autoencoders
• Artificial Intelligence in Drug Discovery: Applications and Techniques
• Knowledge Extracted from Recurrent Deep Belief Network for Real Time Deterministic Control
• Modelling Symbolic Music: Beyond the Piano Roll
• Multimodal Transitions for Generative Stochastic Networks
• BandNet: A Neural Network-based, Multi-Instrument Beatles-Style MIDI Music Composition Machine
• Dance Dance Convolution
• Learning Over Long Time Lags
• BachProp: Learning to Compose Music in Multiple Styles
• Live Orchestral Piano, a system for real-time orchestral music generation
• Stochastic Gradient Monomial Gamma Sampler
• Simplified Long Short-term Memory Recurrent Neural Networks: part III
• Bach or Mock? A Grading Function for Chorales in the Style of J.S. Bach
• Artificial Musical Intelligence: A Survey
• Composing Music with Grammar Argumented Neural Networks and Note-Level Encoding
• Exploiting random projections and sparsity with random forests and gradient boosting methods -- Application to multi-label and multi-output learning, random forest model compression and leveraging input sparsity
• Polyphonic Music Composition with LSTM Neural Networks and Reinforcement Learning
• Anomaly Detection on Graph Time Series
• Human Social Interaction Modeling Using Temporal Deep Networks
• One Deep Music Representation to Rule Them All? : A comparative analysis of different representation learning strategies
• Factored Temporal Sigmoid Belief Networks for Sequence Learning
• Real-time interactive sequence generation and control with Recurrent Neural Network ensembles
• Early Improving Recurrent Elastic Highway Network
• Computer Assisted Composition with Recurrent Neural Networks
• Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks
• Deep Recurrent Neural Networks for Sequential Phenotype Prediction in Genomics
• Improving Choral Music Separation through Expressive Synthesized Data from Sampled Instruments
• An Attentional Neural Network Architecture for Folk Song Classification
• Filtering Variational Objectives
• Generating Nontrivial Melodies for Music as a Service
• Variational Hyper RNN for Sequence Modeling
• Deep Autotuner: A Data-Driven Approach to Natural-Sounding Pitch Correction for Singing Voice in Karaoke Performances
• Regularizing Recurrent Networks - On Injected Noise and Norm-based Methods
• A Differential Geometry Perspective on Orthogonal Recurrent Models
• The GigaMIDI Dataset with Features for Expressive Music Performance Detection
• An End-to-End Neural Network for Polyphonic Piano Music Transcription
• Off the Beaten Track: Using Deep Learning to Interpolate Between Music Genres
• Deep Segment Hash Learning for Music Generation
• Distributed Vector Representations of Folksong Motifs
• Music Generation with Temporal Structure Augmentation
• Artificial Neural Networks Jamming on the Beat
• Music Generation using Three-layered LSTM
• Classical Music Composition Using State Space Models
• MIDI-Sandwich: Multi-model Multi-task Hierarchical Conditional VAE-GAN networks for Symbolic Single-track Music Generation
• Sequence Generation using Deep Recurrent Networks and Embeddings: A study case in music
• Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding
• Dual-track Music Generation using Deep Learning
• Action-Affect Classification and Morphing using Multi-Task Representation Learning
• Recurrent Estimation of Distributions
• Pathwise Derivatives for Multivariate Distributions
• Re-examination of the Role of Latent Variables in Sequence Modeling
• Temporal Pattern Attention for Multivariate Time Series Forecasting
• Robust Implicit Backpropagation
• Deep Generative Networks For Sequence Prediction
• Markov-switching State Space Models for Uncovering Musical Interpretation
• Accelerate RNN-based Training with Importance Sampling
• Towards Machine Intelligence
• A Hybrid Recurrent Neural Network For Music Transcription
• Differential Music: Automated Music Generation Using LSTM Networks with Representation Based on Melodic and Harmonic Intervals
• Harmonic Recomposition using Conditional Autoregressive Modeling
• Modeling Melodic Feature Dependency with Modularized Variational Auto-Encoder
• Temporally Folded Convolutional Neural Networks for Sequence Forecasting