Automatic Hyperparameter Tuning in Sparse Matrix Factorization
arXiv:2305.10114 · doi:10.1162/neco_a_01581
Abstract
We study the problem of hyperparameter tuning in sparse matrix factorization under Bayesian framework. In the prior work, an analytical solution of sparse matrix factorization with Laplace prior was obtained by variational Bayes method under several approximations. Based on this solution, we propose a novel numerical method of hyperparameter tuning by evaluating the zero point of normalization factor in sparse matrix prior. We also verify that our method shows excellent performance for ground-truth sparse matrix reconstruction by comparing it with the widely-used algorithm of sparse principal component analysis.
arXiv comment13 pages, 5 figures
References in corpus (10)
- On the "degrees of freedom" of the lasso
- Phase transitions and sample complexity in Bayes-optimal matrix factorization
- Constrained Low-rank Matrix Estimation: Phase Transitions, Approximate Message Passing and Applications
- Cross validation in LASSO and its acceleration
- Phase Transitions in Sparse PCA
- Phase Diagram and Approximate Message Passing for Blind Calibration and Dictionary Learning
- Statistical Mechanics of Dictionary Learning
- Approximate Leave-One-Out for Fast Parameter Tuning in High Dimensions
- Phase diagram of matrix compressed sensing
- Approximate Method of Variational Bayesian Matrix Factorization/Completion with Sparse Prior