Applied Federated Learning: Improving Google Keyboard Query Suggestions

Abstract

Federated learning is a distributed form of machine learning where both the training data and model training are decentralized. In this paper, we use federated learning in a commercial, global-scale setting to train, evaluate and deploy a model to improve virtual keyboard search suggestion quality without direct access to the underlying user data. We describe our observations in federated training, compare metrics to live deployments, and present resulting quality increases. In whole, we demonstrate how federated learning can be applied end-to-end to both improve user experiences and enhance user privacy.

References in corpus (3)

• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Federated Learning for Mobile Keyboard Prediction
• Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimization

Cited by in corpus (92)

• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• Towards Federated Learning at Scale: System Design
• Robust Aggregation for Federated Learning
• SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
• Sustainable AI: Environmental Implications, Challenges and Opportunities
• Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data
• Three Approaches for Personalization with Applications to Federated Learning
• AI-based Fog and Edge Computing: A Systematic Review, Taxonomy and Future Directions
• Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
• Federated Reinforcement Learning: Techniques, Applications, and Open Challenges
• Federated Learning for Healthcare Domain - Pipeline, Applications and Challenges
• A Field Guide to Federated Optimization
• Efficient Dropout-resilient Aggregation for Privacy-preserving Machine Learning
• Battery-constrained Federated Edge Learning in UAV-enabled IoT for B5G/6G Networks
• Edge Intelligence: Architectures, Challenges, and Applications
• Variational Federated Multi-Task Learning
• Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
• Federated Learning Of Out-Of-Vocabulary Words
• No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices
• Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework
• FetchSGD: Communication-Efficient Federated Learning with Sketching
• Federated Learning Under Intermittent Client Availability and Time-Varying Communication Constraints
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospective
• Communication Efficiency in Federated Learning: Achievements and Challenges
• Communication-Computation Efficient Secure Aggregation for Federated Learning
• Oort: Efficient Federated Learning via Guided Participant Selection
• ProtoSound: A Personalized and Scalable Sound Recognition System for Deaf and Hard-of-Hearing Users
• The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systems
• Federated Learning Meets Natural Language Processing: A Survey
• Generative Models for Effective ML on Private, Decentralized Datasets
• FedScale: Benchmarking Model and System Performance of Federated Learning at Scale
• Auto-weighted Robust Federated Learning with Corrupted Data Sources
• Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies
• Multi-Model Federated Learning
• Hierarchical Federated Learning through LAN-WAN Orchestration
• Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
• FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers
• Secure Federated Submodel Learning
• Practical Vertical Federated Learning with Unsupervised Representation Learning
• Privacy Preservation in Federated Learning: An insightful survey from the GDPR Perspective
• Vertical Federated Knowledge Transfer via Representation Distillation for Healthcare Collaboration Networks
• Federated Reconstruction: Partially Local Federated Learning
• A Systematic Literature Review on Federated Learning: From A Model Quality Perspective
• Dynamic Fusion based Federated Learning for COVID-19 Detection
• FedJAX: Federated learning simulation with JAX
• Helios: Heterogeneity-Aware Federated Learning with Dynamically Balanced Collaboration
• On the Outsized Importance of Learning Rates in Local Update Methods
• Learning, Computing, and Trustworthiness in Intelligent IoT Environments: Performance-Energy Tradeoffs
• Turning Federated Learning Systems Into Covert Channels
• Shielding Federated Learning Systems against Inference Attacks with ARM TrustZone
• Personalized Federated Deep Learning for Pain Estimation From Face Images
• Stragglers Are Not Disaster: A Hybrid Federated Learning Algorithm with Delayed Gradients
• Blockchain-based Federated Learning for Device Failure Detection in Industrial IoT
• Device Heterogeneity in Federated Learning: A Superquantile Approach
• Decentralized Learning Made Easy with DecentralizePy
• Federated Learning with Superquantile Aggregation for Heterogeneous Data
• Layer-wise Characterization of Latent Information Leakage in Federated Learning
• On the Impact of Device and Behavioral Heterogeneity in Federated Learning
• Lottery Hypothesis based Unsupervised Pre-training for Model Compression in Federated Learning
• Sustainable Federated Learning
• Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Data
• 2CP: Decentralized Protocols to Transparently Evaluate Contributivity in Blockchain Federated Learning Environments
• FedSkel: Efficient Federated Learning on Heterogeneous Systems with Skeleton Gradients Update
• Federated Learning -- Methods, Applications and beyond
• The More, the Better? A Study on Collaborative Machine Learning for DGA Detection
• Stochastic Channel-Based Federated Learning for Medical Data Privacy Preserving
• DistFL: Distribution-aware Federated Learning for Mobile Scenarios
• Auxo: Efficient Federated Learning via Scalable Client Clustering
• Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learning
• Efficient Privacy Preserving Edge Computing Framework for Image Classification
• Federated Learning of User Authentication Models
• Privacy Preserving Stochastic Channel-Based Federated Learning with Neural Network Pruning
• PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization
• WAFFLE: Watermarking in Federated Learning
• OpenFed: A Comprehensive and Versatile Open-Source Federated Learning Framework
• Rate-Distortion Theoretic Bounds on Generalization Error for Distributed Learning
• Performance Analysis and Characterization of Training Deep Learning Models on Mobile Devices
• Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices
• An Adaptive Clustering Scheme for Client Selections in Communication-Efficient Federated Learning
• Critical Learning Periods in Federated Learning
• Distributed Learning on Heterogeneous Resource-Constrained Devices
• EasyFL: A Low-code Federated Learning Platform For Dummies
• TiFL: A Tier-based Federated Learning System
• Demystifying the Effects of Non-Independence in Federated Learning
• Boosting Resource-Constrained Federated Learning Systems with Guessed Updates
• FedECA: Federated External Control Arms for Causal Inference with Time-To-Event Data in Distributed Settings
• Data-Free Evaluation of User Contributions in Federated Learning
• Loosely Coupled Federated Learning Over Generative Models
• Management of Resource at the Network Edge for Federated Learning
• Privacy enabled Financial Text Classification using Differential Privacy and Federated Learning