Regularized Least-Mean-Square Algorithms

Abstract

We consider adaptive system identification problems with convex constraints and propose a family of regularized Least-Mean-Square (LMS) algorithms. We show that with a properly selected regularization parameter the regularized LMS provably dominates its conventional counterpart in terms of mean square deviations. We establish simple and closed-form expressions for choosing this regularization parameter. For identifying an unknown sparse system we propose sparse and group-sparse LMS algorithms, which are special examples of the regularized LMS family. Simulation results demonstrate the advantages of the proposed filters in both convergence rate and steady-state error under sparsity assumptions on the true coefficient vector.

9 pages, double column, submitted to IEEE Transactions on Signal Processing

References in corpus (2)

• Consistency of the group Lasso and multiple kernel learning
• Online Sparse System Identification and Signal Reconstruction using Projections onto Weighted ${\ell }_1$ Balls

Cited by in corpus (14)

• A Survey of Stochastic Simulation and Optimization Methods in Signal Processing
• Block-Sparsity-Induced Adaptive Filter for Multi-Clustering System Identification
• Recursive ${\ell }_{1,\infty }$ Group lasso
• Zero Attracting PNLMS Algorithm and Its Convergence in Mean
• A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation
• Understanding and Monitoring Human Trafficking via Social Sensors: A Sociological Approach
• Adaptive Parameters Adjustment for Group Reweighted Zero-Attracting LMS
• r-Instance Learning for Missing People Tweets Identification
• Convergence Analysis of l0-RLS Adaptive Filter
• Study of Sparsity-Aware Subband Adaptive Filtering Algorithms with Adjustable Penalties
• Study of Proximal Normalized Subband Adaptive Algorithm for Acoustic Echo Cancellation
• Olfactory Signal Processing
• Dynamic Topology Adaptation and Distributed Estimation for Smart Grids
• Dynamic Topology Adaptation Based on Adaptive Link Selection Algorithms for Distributed Estimation