Data Augmenting Contrastive Learning of Speech Representations in the Time Domain

Abstract

Contrastive Predictive Coding (CPC), based on predicting future segments of speech based on past segments is emerging as a powerful algorithm for representation learning of speech signal. However, it still under-performs other methods on unsupervised evaluation benchmarks. Here, we introduce WavAugment, a time-domain data augmentation library and find that applying augmentation in the past is generally more efficient and yields better performances than other methods. We find that a combination of pitch modification, additive noise and reverberation substantially increase the performance of CPC (relative improvement of 18-22%), beating the reference Libri-light results with 600 times less data. Using an out-of-domain dataset, time-domain data augmentation can push CPC to be on par with the state of the art on the Zero Speech Benchmark 2017. We also show that time-domain data augmentation consistently improves downstream limited-supervision phoneme classification tasks by a factor of 12-15% relative.

References in corpus (11)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Libri-Light: A Benchmark for ASR with Limited or No Supervision
• Unsupervised speech representation learning using WaveNet autoencoders
• Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules
• Putting An End to End-to-End: Gradient-Isolated Learning of Representations
• Unsupervised Pre-training of Bidirectional Speech Encoders via Masked Reconstruction
• Contrastive Predictive Coding Based Feature for Automatic Speaker Verification
• The Zero Resource Speech Challenge 2020: Discovering discrete subword and word units

Cited by in corpus (7)

• Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challenge
• Text-Free Prosody-Aware Generative Spoken Language Modeling
• Towards Attention-based Contrastive Learning for Audio Spoof Detection
• On a time-frequency blurring operator with applications in data augmentation
• Using Self-Supervised Feature Extractors with Attention for Automatic COVID-19 Detection from Speech
• Exploiting Single-Channel Speech For Multi-channel End-to-end Speech Recognition
• Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering