Learning Audio - Sheet Music Correspondences for Score Identification and Offline Alignment
arXiv:1707.09887
Abstract
This work addresses the problem of matching short excerpts of audio with their respective counterparts in sheet music images. We show how to employ neural network-based cross-modality embedding spaces for solving the following two sheet music-related tasks: retrieving the correct piece of sheet music from a database when given a music audio as a search query; and aligning an audio recording of a piece with the corresponding images of sheet music. We demonstrate the feasibility of this in experiments on classical piano music by five different composers (Bach, Haydn, Mozart, Beethoven and Chopin), and additionally provide a discussion on why we expect multi-modal neural networks to be a fruitful paradigm for dealing with sheet music and audio at the same time.
arXiv commentIn Proceedings of the 18th International Society for Music Information Retrieval Conference (ISMIR 2017)
References in corpus (5)
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
- Metric distances derived from cosine similarity and Pearson and Spearman correlations
- On the Potential of Simple Framewise Approaches to Piano Transcription
- Towards Score Following in Sheet Music Images
Cited by in corpus (5)
- Cross-Modal Music Retrieval and Applications: An Overview of Key Methodologies
- Multimodal music information processing and retrieval: survey and future challenges
- Towards Linking the Lakh and IMSLP Datasets
- A Hybrid Approach to Audio-to-Score Alignment
- Unsupervised Generative Adversarial Alignment Representation for Sheet music, Audio and Lyrics