Population Based Training for Data Augmentation and Regularization in Speech Recognition
arXiv:2010.03899
Abstract
Varying data augmentation policies and regularization over the course of optimization has led to performance improvements over using fixed values. We show that population based training is a useful tool to continuously search those hyperparameters, within a fixed budget. This greatly simplifies the experimental burden and computational cost of finding such optimal schedules. We experiment in speech recognition by optimizing SpecAugment this way, as well as dropout. It compares favorably to a baseline that does not change those hyperparameters over the course of training, with an 8% relative WER improvement. We obtain 5.18% word error rate on LibriSpeech's test-other.
tech report from Dec. 2019
References in corpus (7)
- SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Reducing Transformer Depth on Demand with Structured Dropout
- Transformer-based Acoustic Modeling for Hybrid Speech Recognition
- wav2letter++: The Fastest Open-source Speech Recognition System
- End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
- Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play