Decentralized Sparse Multitask RLS over Networks
arXiv:1702.03366 · doi:10.1109/TSP.2017.2750110
Abstract
Distributed adaptive signal processing has attracted much attention in the recent decade owing to its effectiveness in many decentralized real-time applications in networked systems. Because many natural signals are highly sparse with most entries equal to zero, several decentralized sparse adaptive algorithms have been proposed recently. Most of them is focused on the single task estimation problems, in which all nodes receive data associated with the same unknown vector and collaborate to estimate it. However, many applications are inherently multitask oriented and each node has its own unknown vector different from others'. The related multitask estimation problem benefits from collaborations among the nodes as neighbor nodes usually share analogous properties and thus similar unknown vectors. In this work, we study the distributed sparse multitask recursive least squares (RLS) problem over networks. We first propose a decentralized online alternating direction method of multipliers (ADMM) algorithm for the formulated RLS problem. The algorithm is simplified for easy implementation with closed-form computations in each iteration and low storage requirements. Moreover, to further reduce the complexity, we present a decentralized online subgradient method with low computational overhead. We theoretically analyze the convergence behavior of the proposed subgradient method and derive an error bound related to the network topology and algorithm parameters. The effectiveness of the proposed algorithms is corroborated by numerical simulations and an accuracy-complexity tradeoff between the proposed two algorithms is highlighted.
References in corpus (17)
- On the Linear Convergence of the ADMM in Decentralized Consensus Optimization
- Multitask Diffusion Adaptation over Networks
- Distributed Robust Multi-Cell Coordinated Beamforming with Imperfect CSI: An ADMM Approach
- Diffusion LMS over Multitask Networks
- Sparse Distributed Learning Based on Diffusion Adaptation
- Performance Analysis of l_0 Norm Constraint Least Mean Square Algorithm
- A stochastic gradient approach on compressive sensing signal reconstruction based on adaptive filtering framework
- A Proximal Dual Consensus ADMM Method for Multi-Agent Constrained Optimization
- DQM: Decentralized Quadratically Approximated Alternating Direction Method of Multipliers
- Distributed Clustering and Learning Over Networks
- Online Alternating Direction Method
- A Sparsity-Aware Adaptive Algorithm for Distributed Learning
- Distributed Diffusion-Based LMS for Node-Specific Adaptive Parameter Estimation
- Proximity Without Consensus in Online Multi-Agent Optimization
- Proximal Multitask Learning over Networks with Sparsity-inducing Coregularization
- Diffusion LMS for Multitask Problems with Local Linear Equality Constraints
- Distributed Adaptive Networks: A Graphical Evolutionary Game-Theoretic View