Singular-Value-Decomposition Analysis of Associative Memory in a Neural Network

Abstract

We evaluate performance of associative memory in a neural network by based on the singular value decomposition (SVD) of image data stored in the network. We consider the situation in which the original image and its highly coarse-grained one by SVD are stored in the network and the intermediate one is taken as an input. We find that the performance is characterized by the snapshot-entropy scaling inherent in the SVD: the network retrieves the original image when the entropy of the input image is larger than the critical value determined from the scaling. The result indicates efficiency of the SVD as a criterion of the performance and also indicates universality of the scaling for realistic problems beyond theoretical physics.

6 pages, 5 figures

References in corpus (5)

• Scaling of entanglement support for Matrix Product States
• Advanced Mean Field Theory of Restricted Boltzmann Machine
• Exact Mapping from Singular Value Spectrum of Fractal Images to Entanglement Spectrum of One-Dimensional Quantum Systems
• Random Fractal Ansatz for the configurations of Two-Dimensional Critical Systems
• Inverse Mellin Transformation of Continuous Singular Value Decomposition: A Route to Holographic Renormalization