Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challenge
arXiv:2210.15759 · doi:10.1109/JSTSP.2022.3206084
Abstract
Recent progress in self-supervised or unsupervised machine learning has opened the possibility of building a full speech processing system from raw audio without using any textual representations or expert labels such as phonemes, dictionaries or parse trees. The contribution of the Zero Resource Speech Challenge series since 2015 has been to break down this long-term objective into four well-defined tasks -- Acoustic Unit Discovery, Spoken Term Discovery, Discrete Resynthesis, and Spoken Language Modeling -- and introduce associated metrics and benchmarks enabling model comparison and cumulative progress. We present an overview of the six editions of this challenge series since 2015, discuss the lessons learned, and outline the areas which need more work or give puzzling results.
References in corpus (21)
- Representation Learning with Contrastive Predictive Coding
- WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
- Libri-Light: A Benchmark for ASR with Limited or No Supervision
- Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner
- Unsupervised speech representation learning using WaveNet autoencoders
- Representations of language in a model of visually grounded speech signal
- Masked Autoencoders that Listen
- A segmental framework for fully-unsupervised large-vocabulary speech recognition
- Data Augmenting Contrastive Learning of Speech Representations in the Time Domain
- The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modeling
- Unsupervised End-to-End Learning of Discrete Linguistic Units for Voice Conversion
- Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring
- Contrastive prediction strategies for unsupervised segmentation and categorization of phonemes and words
- Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling
- Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling
- Probing phoneme, language and speaker information in unsupervised speech representations
- Exploring TTS without T Using Biologically/Psychologically Motivated Neural Network Modules (ZeroSpeech 2020)
- Predicting non-native speech perception using the Perceptual Assimilation Model and state-of-the-art acoustic models
- Zero resource speech synthesis using transcripts derived from perceptual acoustic units
- A Brief Overview of Unsupervised Neural Speech Representation Learning
- A Hierarchical Subspace Model for Language-Attuned Acoustic Unit Discovery