Diffusion LMS over Multitask Networks

Abstract

The diffusion LMS algorithm has been extensively studied in recent years. This efficient strategy allows to address distributed optimization problems over networks in the case where nodes have to collaboratively estimate a single parameter vector. Problems of this type are referred to as single-task problems. Nevertheless, there are several problems in practice that are multitask-oriented in the sense that the optimum parameter vector may not be the same for every node. This brings up the issue of studying the performance of the diffusion LMS algorithm when it is run, either intentionally or unintentionally, in a multitask environment. In this paper, we conduct a theoretical analysis on the stochastic behavior of diffusion LMS in the case where the so-called single-task hypothesis is violated. We explain under what conditions diffusion LMS continues to deliver performance superior to non-cooperative strategies in the multitask environment. When the conditions are violated, we explain how to endow the nodes with the ability to cluster with other similar nodes to remove bias. We propose an unsupervised clustering strategy that allows each node to select, via adaptive adjustments of combination weights, the neighboring nodes with which it can collaborate to estimate a common parameter vector. Simulations are presented to illustrate the theoretical results, and to demonstrate the efficiency of the proposed clustering strategy. The framework is applied to a useful problem involving a multi-target tracking task.

30 pages, 14 figures. To appear in IEEE Trans. Signal Process., 2015

References in corpus (4)

• Multitask Diffusion Adaptation over Networks
• Distributed Consensus Algorithms in Sensor Networks: Link Failures and Channel Noise
• Estimation of Space-Time Varying Parameters Using a Diffusion LMS Algorithm
• Distributed dictionary learning over a sensor network

Cited by in corpus (28)

• Distributed Clustering and Learning Over Networks
• Distributed Learning for Stochastic Generalized Nash Equilibrium Problems
• Multitask learning over graphs: An Approach for Distributed, Streaming Machine Learning
• Proximal Multitask Learning over Networks with Sparsity-inducing Coregularization
• Distributed Adaptive Learning of Graph Signals
• Diffusion LMS for Multitask Problems with Local Linear Equality Constraints
• Multitask diffusion adaptation over networks with common latent representations
• On reducing the communication cost of the diffusion LMS algorithm
• Resilient Distributed Parameter Estimation with Heterogeneous Data
• Adaptive Diffusion Schemes for Heterogeneous Networks
• Resilient Distributed Diffusion in Networks with Adversaries
• Decentralized Clustering and Linking by Networked Agents
• Decentralized Sparse Multitask RLS over Networks
• Exact Diffusion for Distributed Optimization and Learning --- Part I: Algorithm Development
• A Multitask Diffusion Strategy with Optimized Inter-Cluster Cooperation
• Asynchronous adaptive networks
• Distributed Multi-task APA over Adaptive Networks Based on Partial Diffusion
• Byzantine Resilient Distributed Multi-Task Learning
• Resilient Distributed Diffusion for Multi-task Estimation
• Affine Combination of Diffusion Strategies over Networks
• Unsupervised diffusion-based LMS for node-specific parameter estimation over wireless sensor networks
• Online Distributed Learning over Graphs with Multitask Graph-Filter Models
• Stochastic Subgradient Algorithms for Strongly Convex Optimization over Distributed Networks
• Diffusion probabilistic LMS algorithm
• Adaptive Parameters Adjustment for Group Reweighted Zero-Attracting LMS
• Privacy-Preserving Distributed Projection LMS for Linear Multitask Networks
• On the Asymptotic Bias of the Diffusion-Based Distributed Pareto Optimization
• Distributed interference cancellation in multi-agent scenarios