Neural Audio Fingerprint for High-specific Audio Retrieval based on Contrastive Learning

Abstract

Most of existing audio fingerprinting systems have limitations to be used for high-specific audio retrieval at scale. In this work, we generate a low-dimensional representation from a short unit segment of audio, and couple this fingerprint with a fast maximum inner-product search. To this end, we present a contrastive learning framework that derives from the segment-level search objective. Each update in training uses a batch consisting of a set of pseudo labels, randomly selected original samples, and their augmented replicas. These replicas can simulate the degrading effects on original audio signals by applying small time offsets and various types of distortions, such as background noise and room/microphone impulse responses. In the segment-level search task, where the conventional audio fingerprinting systems used to fail, our system using 10x smaller storage has shown promising results. Our code and dataset are available at \url{https://mimbres.github.io/neural-audio-fp/}.

ICASSP 2021 (accepted)

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• A Simple Framework for Contrastive Learning of Visual Representations
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Improved Regularization of Convolutional Neural Networks with Cutout
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Simultaneous Feature Learning and Hash Coding with Deep Neural Networks
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Now Playing: Continuous low-power music recognition
• Learning to Screen for Fast Softmax Inference on Large Vocabulary Neural Networks