Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework

Abstract

We propose using federated learning, a decentralized on-device learning paradigm, to train speech recognition models. By performing epochs of training on a per-user basis, federated learning must incur the cost of dealing with non-IID data distributions, which are expected to negatively affect the quality of the trained model. We propose a framework by which the degree of non-IID-ness can be varied, consequently illustrating a trade-off between model quality and the computational cost of federated training, which we capture through a novel metric. Finally, we demonstrate that hyper-parameter optimization and appropriate use of variational noise are sufficient to compensate for the quality impact of non-IID distributions, while decreasing the cost.

arXiv commentPaper published at ICASSP 2021

References in corpus (12)

• TensorFlow: A system for large-scale machine learning
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Federated Learning with Non-IID Data
• Sequence Transduction with Recurrent Neural Networks
• Federated Learning for Mobile Keyboard Prediction
• Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
• Applied Federated Learning: Improving Google Keyboard Query Suggestions
• Federated Learning for Emoji Prediction in a Mobile Keyboard
• Adaptive Federated Optimization
• The Non-IID Data Quagmire of Decentralized Machine Learning
• Federated Transfer Learning with Dynamic Gradient Aggregation

Cited by in corpus (12)

• Federated Learning Meets Natural Language Processing: A Survey
• Auto-weighted Robust Federated Learning with Corrupted Data Sources
• Open Challenges in Synthetic Speech Detection
• Privacy attacks for automatic speech recognition acoustic models in a federated learning framework
• Separate but Together: Unsupervised Federated Learning for Speech Enhancement from Non-IID Data
• Personalized Federated Learning with Exact Stochastic Gradient Descent
• Federated Learning in ASR: Not as Easy as You Think
• ILASR: Privacy-Preserving Incremental Learning for Automatic Speech Recognition at Production Scale
• Ed-Fed: A generic federated learning framework with resource-aware client selection for edge devices
• Private Language Model Adaptation for Speech Recognition
• RecUP-FL: Reconciling Utility and Privacy in Federated Learning via User-configurable Privacy Defense
• Incremental Layer-wise Self-Supervised Learning for Efficient Speech Domain Adaptation On Device