Linear Classification of Neural Manifolds with Correlated Variability

Abstract

Understanding how the statistical and geometric properties of neural activity relate to performance is a key problem in theoretical neuroscience and deep learning. Here, we calculate how correlations between object representations affect the capacity, a measure of linear separability. We show that for spherical object manifolds, introducing correlations between centroids effectively pushes the spheres closer together, while introducing correlations between the axes effectively shrinks their radii, revealing a duality between correlations and geometry with respect to the problem of classification. We then apply our results to accurately estimate the capacity of deep network data.

arXiv comment6 pages and 5 figures in main text. 13 pages and 1 figure in supplementary material

References in corpus (16)

• Neural population geometry: An approach for understanding biological and artificial neural networks
• A mathematical theory of semantic development in deep neural networks
• Classification and Geometry of General Perceptual Manifolds
• Modelling the influence of data structure on learning in neural networks: the hidden manifold model
• Robust information propagation through noisy neural circuits
• Linear Readout of Object Manifolds
• Theory of spike timing based neural classifiers
• Quality of internal representation shapes learning performance in feedback neural networks
• Geometric framework to predict structure from function in neural networks
• Capacity-resolution trade-off in the optimal learning of multiple low-dimensional manifolds by attractor neural networks
• Counting the learnable functions of structured data
• Storage of correlated patterns in a perceptron
• Efficiency of local learning rules in threshold-linear associative networks
• Learning Data Manifolds with a Cutting Plane Method
• Capacity of the covariance perceptron
• Soft-margin classification of object manifolds

Cited by in corpus (4)

• A statistical mechanics framework for Bayesian deep neural networks beyond the infinite-width limit
• Inversion dynamics of class manifolds in deep learning reveals tradeoffs underlying generalisation
• Statistical Mechanics of Support Vector Regression
• Simplified derivations for high-dimensional convex learning problems