Federated Learning Meets Natural Language Processing: A Survey
arXiv:2107.12603
Abstract
Federated Learning aims to learn machine learning models from multiple decentralized edge devices (e.g. mobiles) or servers without sacrificing local data privacy. Recent Natural Language Processing techniques rely on deep learning and large pre-trained language models. However, both big deep neural and language models are trained with huge amounts of data which often lies on the server side. Since text data is widely originated from end users, in this work, we look into recent NLP models and techniques which use federated learning as the learning framework. Our survey discusses major challenges in federated natural language processing, including the algorithm challenges, system challenges as well as the privacy issues. We also provide a critical review of the existing Federated NLP evaluation methods and tools. Finally, we highlight the current research gaps and future directions.
arXiv comment19 pages
References in corpus (23)
- Distilling the Knowledge in a Neural Network
- LSTM: A Search Space Odyssey
- Federated Learning with Non-IID Data
- Federated Learning for Mobile Keyboard Prediction
- Ensemble Distillation for Robust Model Fusion in Federated Learning
- Applied Federated Learning: Improving Google Keyboard Query Suggestions
- Personalized Federated Learning: A Meta-Learning Approach
- Hello Edge: Keyword Spotting on Microcontrollers
- Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
- SecureBoost: A Lossless Federated Learning Framework
- One-Shot Federated Learning
- Federated Tensor Factorization for Computational Phenotyping
- Factorization tricks for LSTM networks
- Soft-Label Dataset Distillation and Text Dataset Distillation
- Distilled One-Shot Federated Learning
- Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework
- FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning
- Federated pretraining and fine tuning of BERT using clinical notes from multiple silos
- Understanding Unintended Memorization in Federated Learning
- Preserving Patient Privacy while Training a Predictive Model of In-hospital Mortality
- Communication-Efficient Federated Distillation
- Two-stage Federated Phenotyping and Patient Representation Learning
- Robust Federated Learning with Attack-Adaptive Aggregation