Sparse Distributed Learning Based on Diffusion Adaptation

Abstract

This article proposes diffusion LMS strategies for distributed estimation over adaptive networks that are able to exploit sparsity in the underlying system model. The approach relies on convex regularization, common in compressive sensing, to enhance the detection of sparsity via a diffusive process over the network. The resulting algorithms endow networks with learning abilities and allow them to learn the sparse structure from the incoming data in real-time, and also to track variations in the sparsity of the model. We provide convergence and mean-square performance analysis of the proposed method and show under what conditions it outperforms the unregularized diffusion version. We also show how to adaptively select the regularization parameter. Simulation results illustrate the advantage of the proposed filters for sparse data recovery.

to appear in IEEE Trans. on Signal Processing, 2013

Cited by in corpus (37)

• Multitask Diffusion Adaptation over Networks
• Diffusion LMS over Multitask Networks
• Combinations of Adaptive Filters
• Adaptive Least Mean Squares Estimation of Graph Signals
• FROST -- Fast row-stochastic optimization with uncoordinated step-sizes
• Diffusion LMS Strategies in Sensor Networks with Noisy Input Data
• Proximal Multitask Learning over Networks with Sparsity-inducing Coregularization
• Distributed Adaptive Learning of Graph Signals
• On reducing the communication cost of the diffusion LMS algorithm
• Adaptive Diffusion Schemes for Heterogeneous Networks
• Diffusion Leaky Zero Attracting Least Mean Square Algorithm and Its Performance Analysis
• A Framework for Parallel and Distributed Training of Neural Networks
• Diffusion Adaptation Strategies for Distributed Estimation over Gaussian Markov Random Fields
• Decentralized Sparse Multitask RLS over Networks
• Application of Compressive Sensing Techniques in Distributed Sensor Networks: A Survey
• Greedy Sparsity-Promoting Algorithms for Distributed Learning
• Diffusion leaky LMS algorithm: analysis and implementation
• Distributed inference over directed networks: Performance limits and optimal design
• An Online Parallel and Distributed Algorithm for Recursive Estimation of Sparse Signals
• Fast and Robust Sparsity Learning over Networks: A Decentralized Surrogate Median Regression Approach
• Sparse Distributed Learning via Heterogeneous Diffusion Adaptive Networks
• NEXT: In-Network Nonconvex Optimization
• Asynchronous adaptive networks
• Distributed Stochastic Nonconvex Optimization and Learning based on Successive Convex Approximation
• Affine Combination of Diffusion Strategies over Networks
• Locally Convex Sparse Learning over Networks
• Diffusion Adaptation Framework for Compressive Sensing Reconstruction
• Deterministic and Randomized Diffusion based Iterative Generalized Hard Thresholding (DiFIGHT) for Distributed Sparse Signal Recovery
• Diffusion L0-norm constraint improved proportionate LMS algorithm for sparse distributed estimation
• Distributed Compressed Estimation for Wireless Sensor Networks Based on Compressive Sensing
• Study of Sparsity-Aware Subband Adaptive Filtering Algorithms with Adjustable Penalties
• Study of Distributed Spectrum Estimation Using Alternating Mixed Discrete-Continuous Adaptation
• Distributed Sparse Feature Selection in Communication-Restricted Networks
• Study of Sparsity-Aware Distributed Conjugate Gradient Algorithms for Sensor Networks
• Study of Diffusion Normalized Least Mean M-estimate Algorithms
• Sparsity-Aware SSAF Algorithm with Individual Weighting Factors for Acoustic Echo Cancellation
• Weighted diffusion LMP algorithm for distributed estimation in non-uniform noise conditions