Singular-Value-Decomposition Analysis of Associative Memory in a Neural Network
arXiv:1608.08333 · doi:10.7566/JPSJ.86.024005
Abstract
We evaluate performance of associative memory in a neural network by based on the singular value decomposition (SVD) of image data stored in the network. We consider the situation in which the original image and its highly coarse-grained one by SVD are stored in the network and the intermediate one is taken as an input. We find that the performance is characterized by the snapshot-entropy scaling inherent in the SVD: the network retrieves the original image when the entropy of the input image is larger than the critical value determined from the scaling. The result indicates efficiency of the SVD as a criterion of the performance and also indicates universality of the scaling for realistic problems beyond theoretical physics.
arXiv comment6 pages, 5 figures
References in corpus (19)
- Holographic Derivation of Entanglement Entropy from AdS/CFT
- Entanglement Entropy and Quantum Field Theory
- Geometric and Renormalized Entropy in Conformal Field Theory
- Entanglement renormalization
- Entanglement Renormalization and Holography
- Tensor Network Renormalization
- Theory of finite-entanglement scaling at one-dimensional quantum critical points
- Algorithms for entanglement renormalization
- Scaling of entanglement support for Matrix Product States
- Tensor network renormalization yields the multi-scale entanglement renormalization ansatz
- Density Matrix Renormalization Group of Gapless Systems
- Local scale transformations on the lattice with tensor network renormalization
- Advanced Mean Field Theory of Restricted Boltzmann Machine
- Exact Mapping from Singular Value Spectrum of Fractal Images to Entanglement Spectrum of One-Dimensional Quantum Systems
- Snapshot spectrum and critical phenomenon for two-dimensional classical spin systems
- Entropy of Factorized Snapshot Data for Two-Dimensional Classical Spin Models
- Random Fractal Ansatz for the configurations of Two-Dimensional Critical Systems
- Comment on "Snapshot spectrum and critical phenomenon for two-dimensional classical spin systems"
- Inverse Mellin Transformation of Continuous Singular Value Decomposition: A Route to Holographic Renormalization