Inexact-ADMM Based Federated Meta-Learning for Fast and Continual Edge Learning

Abstract

In order to meet the requirements for performance, safety, and latency in many IoT applications, intelligent decisions must be made right here right now at the network edge. However, the constrained resources and limited local data amount pose significant challenges to the development of edge AI. To overcome these challenges, we explore continual edge learning capable of leveraging the knowledge transfer from previous tasks. Aiming to achieve fast and continual edge learning, we propose a platform-aided federated meta-learning architecture where edge nodes collaboratively learn a meta-model, aided by the knowledge transfer from prior tasks. The edge learning problem is cast as a regularized optimization problem, where the valuable knowledge learned from previous tasks is extracted as regularization. Then, we devise an ADMM based federated meta-learning algorithm, namely ADMM-FedMeta, where ADMM offers a natural mechanism to decompose the original problem into many subproblems which can be solved in parallel across edge nodes and the platform. Further, a variant of inexact-ADMM method is employed where the subproblems are `solved' via linear approximation as well as Hessian estimation to reduce the computational cost per round to $\mathcal{O}(n)$. We provide a comprehensive analysis of ADMM-FedMeta, in terms of the convergence properties, the rapid adaptation performance, and the forgetting effect of prior knowledge transfer, for the general non-convex case. Extensive experimental studies demonstrate the effectiveness and efficiency of ADMM-FedMeta, and showcase that it substantially outperforms the existing baselines.

References in corpus (22)

• Overcoming catastrophic forgetting in neural networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Continual Learning Through Synaptic Intelligence
• Gradient Episodic Memory for Continual Learning
• Improving Federated Learning Personalization via Model Agnostic Meta Learning
• Personalized Federated Learning: A Meta-Learning Approach
• Progress & Compress: A scalable framework for continual learning
• Federated Meta-Learning with Fast Convergence and Efficient Communication
• Less-forgetting Learning in Deep Neural Networks
• Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
• Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems
• ES-MAML: Simple Hessian-Free Meta Learning
• ADMM for Efficient Deep Learning with Global Convergence
• Inexact-ADMM Based Federated Meta-Learning for Fast and Continual Edge Learning
• On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms
• On the Global Optimality of Model-Agnostic Meta-Learning
• Convergence for nonconvex ADMM, with applications to CT imaging
• Meta Continual Learning via Dynamic Programming
• Transfer Learning via ${\ell }_1$ Regularization
• Expectation Learning for Adaptive Crossmodal Stimuli Association

Cited by in corpus (9)

• Inexact-ADMM Based Federated Meta-Learning for Fast and Continual Edge Learning
• Federated PCA on Grassmann Manifold for IoT Anomaly Detection
• Federated Meta Learning Enhanced Acoustic Radio Cooperative Framework for Ocean of Things Underwater Acoustic Communications
• FedGiA: An Efficient Hybrid Algorithm for Federated Learning
• Communication-Efficient ADMM-based Federated Learning
• Personalized Graph Federated Learning with Differential Privacy
• Automated Federated Learning in Mobile Edge Networks -- Fast Adaptation and Convergence
• A Blockchain-based Reliable Federated Meta-learning for Metaverse: A Dual Game Framework
• Federated Deep Equilibrium Learning: Harnessing Compact Global Representations to Enhance Personalization