Asynchronous Federated Optimization

Abstract

Federated learning enables training on a massive number of edge devices. To improve flexibility and scalability, we propose a new asynchronous federated optimization algorithm. We prove that the proposed approach has near-linear convergence to a global optimum, for both strongly convex and a restricted family of non-convex problems. Empirical results show that the proposed algorithm converges quickly and tolerates staleness in various applications.

References in corpus (11)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Towards Federated Learning at Scale: System Design
• Federated Optimization:Distributed Optimization Beyond the Datacenter
• Pointer Sentinel Mixture Models
• Big Data Caching for Networking: Moving from Cloud to Edge
• Slow Learners are Fast
• Asynchronous Stochastic Gradient Descent with Delay Compensation
• Asynchronous Decentralized Parallel Stochastic Gradient Descent

Cited by in corpus (64)

• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• On the Convergence of FedAvg on Non-IID Data
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
• Mobility-Aware Cooperative Caching in Vehicular Edge Computing Based on Asynchronous Federated and Deep Reinforcement Learning
• Federated Learning: Opportunities and Challenges
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks
• Ground-Assisted Federated Learning in LEO Satellite Constellations
• Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey
• Accelerating Federated Learning over Reliability-Agnostic Clients in Mobile Edge Computing Systems
• Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data
• Artificial Intelligence of Things: A Survey
• Distilled One-Shot Federated Learning
• Secure and Efficient Federated Learning Through Layering and Sharding Blockchain
• CSAFL: A Clustered Semi-Asynchronous Federated Learning Framework
• Asynchronous Federated Learning on Heterogeneous Devices: A Survey
• Towards Communication-efficient and Attack-Resistant Federated Edge Learning for Industrial Internet of Things
• FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation
• Semi-Synchronous Federated Learning for Energy-Efficient Training and Accelerated Convergence in Cross-Silo Settings
• FederBoost: Private Federated Learning for GBDT
• Time-sensitive Learning for Heterogeneous Federated Edge Intelligence
• Federated Learning with Buffered Asynchronous Aggregation
• FedHe: Heterogeneous Models and Communication-Efficient Federated Learning
• Papaya: Practical, Private, and Scalable Federated Learning
• GitFL: Adaptive Asynchronous Federated Learning using Version Control
• A Survey on Federated Learning and its Applications for Accelerating Industrial Internet of Things
• FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers
• DYNAMITE: Dynamic Interplay of Mini-Batch Size and Aggregation Frequency for Federated Learning with Static and Streaming Dataset
• Fast Federated Learning in the Presence of Arbitrary Device Unavailability
• Dynamic Fusion based Federated Learning for COVID-19 Detection
• FedGSM: Efficient Federated Learning for LEO Constellations with Gradient Staleness Mitigation
• Helios: Heterogeneity-Aware Federated Learning with Dynamically Balanced Collaboration
• Asynchronous Online Federated Learning for Edge Devices with Non-IID Data
• Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise
• Secure Aggregation for Buffered Asynchronous Federated Learning
• FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization
• FedLesScan: Mitigating Stragglers in Serverless Federated Learning
• Asynchronous Online Federated Learning with Reduced Communication Requirements
• Sample-based and Feature-based Federated Learning for Unconstrained and Constrained Nonconvex Optimization via Mini-batch SSCA
• Towards On-Device Federated Learning: A Direct Acyclic Graph-based Blockchain Approach
• Federated Edge Learning : Design Issues and Challenges
• Anarchic Federated Learning
• Federated Action Recognition on Heterogeneous Embedded Devices
• Returning the Favor: What Wireless Networking Can Offer to AI and Edge Learning
• Robustness of Decentralised Learning to Nodes and Data Disruption
• A Perspective on Time towards Wireless 6G
• Adaptive Task Allocation for Asynchronous Federated and Parallelized Mobile Edge Learning
• Communication-Efficient Zeroth-Order Distributed Online Optimization: Algorithm, Theory, and Applications
• AdaptCL: Efficient Collaborative Learning with Dynamic and Adaptive Pruning
• Task Allocation for Asynchronous Mobile Edge Learning with Delay and Energy Constraints
• Device Scheduling and Update Aggregation Policies for Asynchronous Federated Learning
• Convergence Analysis of Decentralized ASGD
• FLRA: A Reference Architecture for Federated Learning Systems
• Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes
• Dynamic Attention-based Communication-Efficient Federated Learning
• Asynchronous Federated Learning for Sensor Data with Concept Drift
• Accelerating Federated Learning in Heterogeneous Data and Computational Environments
• On the Convergence of Quantized Parallel Restarted SGD for Central Server Free Distributed Training
• Asynchronous Distributed Optimization with Stochastic Delays
• HADFL: Heterogeneity-aware Decentralized Federated Learning Framework
• On Addressing Heterogeneity in Federated Learning for Autonomous Vehicles Connected to a Drone Orchestrator
• InFL-UX: A Toolkit for Web-Based Interactive Federated Learning
• Management of Resource at the Network Edge for Federated Learning