Matrix factorization with neural networks

Abstract

Matrix factorization is an important mathematical problem encountered in the context of dictionary learning, recommendation systems and machine learning. We introduce a new `decimation' scheme that maps it to neural network models of associative memory and provide a detailed theoretical analysis of its performance, showing that decimation is able to factorize extensive-rank matrices and to denoise them efficiently. We introduce a decimation algorithm based on ground-state search of the neural network, which shows performances that match the theoretical prediction.

arXiv comment13 pages, 6 figures

References in corpus (9)

• Message Passing Algorithms for Compressed Sensing
• Phase transitions and sample complexity in Bayes-optimal matrix factorization
• Rotational invariant estimator for general noisy matrices
• The adaptive interpolation method for proving replica formulas. Applications to the Curie-Weiss and Wigner spike models
• Fundamental limits of detection in the spiked Wigner model
• Statistical limits of dictionary learning: random matrix theory and the spectral replica method
• Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising
• Supervised perceptron learning vs unsupervised Hebbian unlearning: Approaching optimal memory retrieval in Hopfield-like networks
• Bayesian reconstruction of memories stored in neural networks from their connectivity

Cited by in corpus (4)

• Matrix denoising: Bayes-optimal estimators via low-degree polynomials
• Some observations on the ambivalent role of symmetries in Bayesian inference problems
• Exact Replica Symmetric solution for transverse field Hopfield model under finite Trotter size
• Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens