Cross-Modal Music Retrieval and Applications: An Overview of Key Methodologies

Abstract

There has been a rapid growth of digitally available music data, including audio recordings, digitized images of sheet music, album covers and liner notes, and video clips. This huge amount of data calls for retrieval strategies that allow users to explore large music collections in a convenient way. More precisely, there is a need for cross-modal retrieval algorithms that, given a query in one modality (e.g., a short audio excerpt), find corresponding information and entities in other modalities (e.g., the name of the piece and the sheet music). This goes beyond exact audio identification and subsequent retrieval of metainformation as performed by commercial applications like Shazam [1].

References in corpus (3)

• Deep Learning in Neural Networks: An Overview
• Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
• Learning Audio - Sheet Music Correspondences for Score Identification and Offline Alignment

Cited by in corpus (10)

• Multimodal music information processing and retrieval: survey and future challenges
• Using Cell Phone Pictures of Sheet Music To Retrieve MIDI Passages
• Towards Linking the Lakh and IMSLP Datasets
• Improved Handling of Repeats and Jumps in Audio-Sheet Image Synchronization
• Multi-Modal Music Information Retrieval: Augmenting Audio-Analysis with Visual Computing for Improved Music Video Analysis
• Learning Soft-Attention Models for Tempo-invariant Audio-Sheet Music Retrieval
• Camera-Based Piano Sheet Music Identification
• MIDI Passage Retrieval Using Cell Phone Pictures of Sheet Music
• Learning Explicit and Implicit Latent Common Spaces for Audio-Visual Cross-Modal Retrieval
• Audio-based Musical Version Identification: Elements and Challenges