Matrix factorization with neural networks

Abstract

Matrix factorization is an important mathematical problem encountered in the context of dictionary learning, recommendation systems and machine learning. We introduce a new `decimation' scheme that maps it to neural network models of associative memory and provide a detailed theoretical analysis of its performance, showing that decimation is able to factorize extensive-rank matrices and to denoise them efficiently. We introduce a decimation algorithm based on ground-state search of the neural network, which shows performances that match the theoretical prediction.

13 pages, 6 figures

References in corpus (4)

• Statistical limits of dictionary learning: random matrix theory and the spectral replica method
• Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising
• Supervised perceptron learning vs unsupervised Hebbian unlearning: Approaching optimal memory retrieval in Hopfield-like networks
• Bayesian reconstruction of memories stored in neural networks from their connectivity

Cited by in corpus (4)

• Matrix denoising: Bayes-optimal estimators via low-degree polynomials
• Some observations on the ambivalent role of symmetries in Bayesian inference problems
• Exact Replica Symmetric solution for transverse field Hopfield model under finite Trotter size
• Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens