Diffusion Strategies Outperform Consensus Strategies for Distributed Estimation over Adaptive Networks
arXiv:1205.3993 · doi:10.1109/TSP.2012.2217338
Abstract
Adaptive networks consist of a collection of nodes with adaptation and learning abilities. The nodes interact with each other on a local level and diffuse information across the network to solve estimation and inference tasks in a distributed manner. In this work, we compare the mean-square performance of two main strategies for distributed estimation over networks: consensus strategies and diffusion strategies. The analysis in the paper confirms that under constant step-sizes, diffusion strategies allow information to diffuse more thoroughly through the network and this property has a favorable effect on the evolution of the network: diffusion networks are shown to converge faster and reach lower mean-square deviation than consensus networks, and their mean-square stability is insensitive to the choice of the combination weights. In contrast, and surprisingly, it is shown that consensus networks can become unstable even if all the individual nodes are stable and able to solve the estimation task on their own. When this occurs, cooperation over the network leads to a catastrophic failure of the estimation task. This phenomenon does not occur for diffusion networks: we show that stability of the individual nodes always ensures stability of the diffusion network irrespective of the combination topology. Simulation results support the theoretical findings.
arXiv comment37 pages, 7 figures, To appear in IEEE Transactions on Signal Processing, 2012
References in corpus (8)
- Gossip Algorithms for Distributed Signal Processing
- Diffusion Adaptation Strategies for Distributed Optimization and Learning over Networks
- Diffusion Strategies Outperform Consensus Strategies for Distributed Estimation over Adaptive Networks
- Sensor Networks with Random Links: Topology Design for Distributed Consensus
- Convergence Rate Analysis of Distributed Gossip (Linear Parameter) Estimation: Fundamental Limits and Tradeoffs
- Diffusion Adaptation over Networks under Imperfect Information Exchange and Non-stationary Data
- Weight Optimization for Consensus Algorithms with Correlated Switching Topology
- Large Deviations Performance of Consensus+Innovations Distributed Detection with Non-Gaussian Observations
Cited by in corpus (62)
- Diffusion Strategies Outperform Consensus Strategies for Distributed Estimation over Adaptive Networks
- Multitask Diffusion Adaptation over Networks
- Diffusion LMS over Multitask Networks
- Distributed Random Projection Algorithm for Convex Optimization
- Performance Limits for Distributed Estimation Over LMS Adaptive Networks
- Distributed Clustering and Learning Over Networks
- Adaptive Penalty-Based Distributed Stochastic Convex Optimization
- Distributed Learning for Stochastic Generalized Nash Equilibrium Problems
- Dictionary Learning over Distributed Models
- Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big, Distributed Data
- Compressive Diffusion Strategies Over Distributed Networks for Reduced Communication Load
- On the genericity properties in networked estimation: Topology design and sensor placement
- Diffusion LMS Strategies in Sensor Networks with Noisy Input Data
- Proximal Multitask Learning over Networks with Sparsity-inducing Coregularization
- Study of Robust Diffusion Recursive Least Squares Algorithms with Side Information for Networked Agents
- Multitask diffusion adaptation over networks with common latent representations
- Diffusion-Based Adaptive Distributed Detection: Steady-State Performance in the Slow Adaptation Regime
- Stability and Performance Limits of Adaptive Primal-Dual Networks
- Estimation of Space-Time Varying Parameters Using a Diffusion LMS Algorithm
- Distributed Decision-Making over Adaptive Networks
- Graphic-theoretic distributed inference in social networks
- On the Influence of Informed Agents on Learning and Adaptation over Networks
- Distributed Estimation Recovery under Sensor Failure
- Diffusion Adaptation over Multi-Agent Networks with Wireless Link Impairments
- Diffusion LMS with Communication Delays: Stability and Performance Analysis
- On Distributed Online Classification in the Midst of Concept Drifts
- Distributed Local Linear Parameter Estimation using Gaussian SPAWN
- Multi-hop Diffusion LMS for Energy-constrained Distributed Estimation
- Diffusion Adaptation Strategies for Distributed Estimation over Gaussian Markov Random Fields
- Distributed Stochastic Approximation: Weak Convergence and Network Design
- Diffusion Estimation Over Cooperative Multi-Agent Networks With Missing Data
- Structural cost-optimal design of sensor networks for distributed estimation
- Accelerated Distributed Dual Averaging over Evolving Networks of Growing Connectivity
- Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advances
- Coordinate-Descent Diffusion Learning by Networked Agents
- A Multitask Diffusion Strategy with Optimized Inter-Cluster Cooperation
- Distributed Universal Adaptive Networks
- Distributed Least-Squares Iterative Methods in Networks: A Survey
- Asynchronous Adaptation and Learning over Networks --- Part I: Modeling and Stability Analysis
- Distributed Multi-task APA over Adaptive Networks Based on Partial Diffusion
- Asynchronous adaptive networks
- Performance Limits of Stochastic Sub-Gradient Learning, Part II: Multi-Agent Case
- Information Exchange and Learning Dynamics over Weakly-Connected Adaptive Networks
- Affine Combination of Diffusion Strategies over Networks
- Stochastic Analysis of the Diffusion Least Mean Square and Normalized Least Mean Square Algorithms for Cyclostationary White Gaussian and Non-Gaussian Inputs
- Reinforcement Learning and Nonparametric Detection of Game-Theoretic Equilibrium Play in Social Networks
- Diffusion probabilistic LMS algorithm
- Deterministic and Randomized Diffusion based Iterative Generalized Hard Thresholding (DiFIGHT) for Distributed Sparse Signal Recovery
- Cloud-aided collaborative estimation by ADMM-RLS algorithms for connected vehicle prognostics
- Stochastic Subgradient Algorithms for Strongly Convex Optimization over Distributed Networks
- Dynamic Average Diffusion with randomized Coordinate Updates
- Diffusion LMS for clustered multitask networks
- When Adaptive Diffusion Algorithm Converges to True Parameter?
- Study of Diffusion Normalized Least Mean M-estimate Algorithms
- Distributed interference cancellation in multi-agent scenarios
- Study of Robust Distributed Diffusion RLS Algorithms with Side Information for Adaptive Networks
- Distributed Estimation using Bayesian Consensus Filtering
- Diffusion L0-norm constraint improved proportionate LMS algorithm for sparse distributed estimation
- An Improved Self-Organizing Diffusion Mobile Adaptive Network for Pursuing a Target
- Distributed Algorithms for Aggregative Games on Graphs
- Team-Optimal Distributed MMSE Estimation in General and Tree Networks
- Distributed Policy Evaluation Under Multiple Behavior Strategies