Learning Model-Based Sparsity via Projected Gradient Descent

Abstract

Several convex formulation methods have been proposed previously for statistical estimation with structured sparsity as the prior. These methods often require a carefully tuned regularization parameter, often a cumbersome or heuristic exercise. Furthermore, the estimate that these methods produce might not belong to the desired sparsity model, albeit accurately approximating the true parameter. Therefore, greedy-type algorithms could often be more desirable in estimating structured-sparse parameters. So far, these greedy methods have mostly focused on linear statistical models. In this paper we study the projected gradient descent with non-convex structured-sparse parameter model as the constraint set. Should the cost function have a Stable Model-Restricted Hessian the algorithm produces an approximation for the desired minimizer. As an example we elaborate on application of the main results to estimation in Generalized Linear Model.

References in corpus (5)

• Structured Compressed Sensing: From Theory to Applications
• Consistency of the group Lasso and multiple kernel learning
• Greedy-Like Algorithms for the Cosparse Analysis Model
• Honest variable selection in linear and logistic regression models via ${\ell }_1$ and ${\ell }_1+{\ell }_2$ penalization
• Sublinear Time, Approximate Model-based Sparse Recovery For All

Cited by in corpus (6)

• Deep Learning for Video Classification and Captioning
• Array Resource Allocation for Radar and Communication Integration Network
• Global and Quadratic Convergence of Newton Hard-Thresholding Pursuit
• Technical Report: Graph-Structured Sparse Optimization for Connected Subgraph Detection
• Technical Report: A Generalized Matching Pursuit Approach for Graph-Structured Sparsity
• A Generic Framework for Interesting Subspace Cluster Detection in Multi-attributed Networks