Distributed Clustering and Learning Over Networks

Abstract

Distributed processing over networks relies on in-network processing and cooperation among neighboring agents. Cooperation is beneficial when agents share a common objective. However, in many applications agents may belong to different clusters that pursue different objectives. Then, indiscriminate cooperation will lead to undesired results. In this work, we propose an adaptive clustering and learning scheme that allows agents to learn which neighbors they should cooperate with and which other neighbors they should ignore. In doing so, the resulting algorithm enables the agents to identify their clusters and to attain improved learning and estimation accuracy over networks. We carry out a detailed mean-square analysis and assess the error probabilities of Types I and II, i.e., false alarm and mis-detection, for the clustering mechanism. Among other results, we establish that these probabilities decay exponentially with the step-sizes so that the probability of correct clustering can be made arbitrarily close to one.

47 pages, 6 figures

References in corpus (5)

• Multitask Diffusion Adaptation over Networks
• Clustered Multi-Task Learning: A Convex Formulation
• Diffusion LMS over Multitask Networks
• Distributed Diffusion-Based LMS for Node-Specific Adaptive Parameter Estimation
• Distributed Diffusion-based LMS for Node-Specific Parameter Estimation over Adaptive Networks

Cited by in corpus (20)

• Multitask learning over graphs: An Approach for Distributed, Streaming Machine Learning
• Proximal Multitask Learning over Networks with Sparsity-inducing Coregularization
• Diffusion LMS for Multitask Problems with Local Linear Equality Constraints
• Multitask diffusion adaptation over networks with common latent representations
• Resilient Distributed Diffusion in Networks with Adversaries
• Communication-Censored Linearized ADMM for Decentralized Consensus Optimization
• Decentralized Clustering and Linking by Networked Agents
• Decentralized Sparse Multitask RLS over Networks
• Exact Diffusion for Distributed Optimization and Learning --- Part I: Algorithm Development
• A Multitask Diffusion Strategy with Optimized Inter-Cluster Cooperation
• Walkman: A Communication-Efficient Random-Walk Algorithm for Decentralized Optimization
• Linear Regression with Distributed Learning: A Generalization Error Perspective
• Exact Diffusion for Distributed Optimization and Learning --- Part II: Convergence Analysis
• Resilient Distributed Diffusion for Multi-task Estimation
• Information Exchange and Learning Dynamics over Weakly-Connected Adaptive Networks
• Stochastic Subgradient Algorithms for Strongly Convex Optimization over Distributed Networks
• Unsupervised diffusion-based LMS for node-specific parameter estimation over wireless sensor networks
• Online Distributed Learning over Graphs with Multitask Graph-Filter Models
• Privacy-Preserving Distributed Projection LMS for Linear Multitask Networks
• An Improved Self-Organizing Diffusion Mobile Adaptive Network for Pursuing a Target