Perceptron capacity revisited: classification ability for correlated patterns

Abstract

In this paper, we address the problem of how many randomly labeled patterns can be correctly classified by a single-layer perceptron when the patterns are correlated with each other. In order to solve this problem, two analytical schemes are developed based on the replica method and Thouless-Anderson-Palmer (TAP) approach by utilizing an integral formula concerning random rectangular matrices. The validity and relevance of the developed methodologies are shown for one known result and two example problems. A message-passing algorithm to perform the TAP scheme is also presented.

References in corpus (5)

• Analysis of CDMA systems that are characterized by eigenvalue spectrum
• Inference from correlated patterns: a unified theory for perceptron learning and linear vector channels
• Statistical mechanics of lossy compression using multilayer perceptrons
• Statistical mechanical analysis of the linear vector channel in digital communication
• Gene-network inference by message passing

Cited by in corpus (15)

• Statistical physics of inference: Thresholds and algorithms
• Origin of the computational hardness for learning with binary synapses
• High-temperature Expansions and Message Passing Algorithms
• Mean-field inference methods for neural networks
• Activation function dependence of the storage capacity of treelike neural networks
• Teacher-student learning for a binary perceptron with quantum fluctuations
• Adaptive Thouless-Anderson-Palmer approach to inverse Ising problems with quenched random fields
• Learning of correlated patterns by simple perceptrons
• Self-Averaging Property of Minimal Investment Risk of Mean-Variance Model
• Minimal Investment Risk of Portfolio Optimization Problem with Budget and Investment Concentration Constraints
• Macroscopic Analysis of Vector Approximate Message Passing in a Model Mismatch Setting
• Belief Propagation Algorithm for Portfolio Optimization Problems
• Portfolio Optimization Problem with Non-identical Variances of Asset Returns using Statistical Mechanical Informatics
• Typical $l_1$-recovery limit of sparse vectors represented by concatenations of random orthogonal matrices
• Optimal Learning with Excitatory and Inhibitory synapses