Multitask Diffusion Adaptation over Networks

Abstract

Adaptive networks are suitable for decentralized inference tasks, e.g., to monitor complex natural phenomena. Recent research works have intensively studied distributed optimization problems in the case where the nodes have to estimate a single optimum parameter vector collaboratively. However, there are many important applications that are multitask-oriented in the sense that there are multiple optimum parameter vectors to be inferred simultaneously, in a collaborative manner, over the area covered by the network. In this paper, we employ diffusion strategies to develop distributed algorithms that address multitask problems by minimizing an appropriate mean-square error criterion with ${\ell }_2$-regularization. The stability and convergence of the algorithm in the mean and in the mean-square sense is analyzed. Simulations are conducted to verify the theoretical findings, and to illustrate how the distributed strategy can be used in several useful applications related to spectral sensing, target localization, and hyperspectral data unmixing.

29 pages, 11 figures, submitted for publication

References in corpus (1)

• Adaptive Penalty-Based Distributed Stochastic Convex Optimization

Cited by in corpus (42)

• Diffusion LMS over Multitask Networks
• Distributed Clustering and Learning Over Networks
• Distributed Learning for Stochastic Generalized Nash Equilibrium Problems
• Multitask learning over graphs: An Approach for Distributed, Streaming Machine Learning
• Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big, Distributed Data
• Distributed Diffusion-Based LMS for Node-Specific Adaptive Parameter Estimation
• Online Distributed Learning Over Networks in RKH Spaces Using Random Fourier Features
• Proximity Without Consensus in Online Multi-Agent Optimization
• Proximal Multitask Learning over Networks with Sparsity-inducing Coregularization
• Distributed Adaptive Learning of Graph Signals
• Diffusion LMS for Multitask Problems with Local Linear Equality Constraints
• Multitask diffusion adaptation over networks with common latent representations
• On reducing the communication cost of the diffusion LMS algorithm
• Resilient Distributed Diffusion in Networks with Adversaries
• Adaptation and learning over networks under subspace constraints -- Part I: Stability Analysis
• Quantization for decentralized learning under subspace constraints
• Diffusion LMS with Communication Delays: Stability and Performance Analysis
• Diffusion Leaky Zero Attracting Least Mean Square Algorithm and Its Performance Analysis
• Decentralized Clustering and Linking by Networked Agents
• Decentralized Sparse Multitask RLS over Networks
• Transient Theoretical Analysis of Diffusion RLS Algorithm for Cyclostationary Colored Inputs
• Personalized Graph Federated Learning with Differential Privacy
• A Multitask Diffusion Strategy with Optimized Inter-Cluster Cooperation
• Tracking Performance of Online Stochastic Learners
• DJAM: distributed Jacobi asynchronous method for learning personal models
• Byzantine Resilient Distributed Multi-Task Learning
• Distributed Multi-task APA over Adaptive Networks Based on Partial Diffusion
• Asynchronous adaptive networks
• Resilient Distributed Diffusion for Multi-task Estimation
• Asynchronous Decentralized Stochastic Optimization in Heterogeneous Networks
• Affine Combination of Diffusion Strategies over Networks
• Information Exchange and Learning Dynamics over Weakly-Connected Adaptive Networks
• Distributed Unmixing of Hyperspectral Data With Sparsity Constraint
• Unsupervised diffusion-based LMS for node-specific parameter estimation over wireless sensor networks
• Learning an Interpretable Graph Structure in Multi-Task Learning
• Stochastic Subgradient Algorithms for Strongly Convex Optimization over Distributed Networks
• Distributed Networked Real-time Learning
• Distributed Networked Learning with Correlated Data
• Sparsity Constrained Distributed Unmixing of Hyperspectral Data
• Privacy-Preserving Distributed Projection LMS for Linear Multitask Networks
• Distributed Linearized ADMM for Network Cost Minimization
• Stochastic Behavior of the Nonnegative Least Mean Fourth Algorithm for Stationary Gaussian Inputs and Slow Learning