Classification and Geometry of General Perceptual Manifolds

Abstract

Perceptual manifolds arise when a neural population responds to an ensemble of sensory signals associated with different physical features (e.g., orientation, pose, scale, location, and intensity) of the same perceptual object. Object recognition and discrimination requires classifying the manifolds in a manner that is insensitive to variability within a manifold. How neuronal systems give rise to invariant object classification and recognition is a fundamental problem in brain theory as well as in machine learning. Here we study the ability of a readout network to classify objects from their perceptual manifold representations. We develop a statistical mechanical theory for the linear classification of manifolds with arbitrary geometry revealing a remarkable relation to the mathematics of conic decomposition. Novel geometrical measures of manifold radius and manifold dimension are introduced which can explain the classification capacity for manifolds of various geometries. The general theory is demonstrated on a number of representative manifolds, including L2 ellipsoids prototypical of strictly convex manifolds, L1 balls representing polytopes consisting of finite sample points, and orientation manifolds which arise from neurons tuned to respond to a continuous angle variable, such as object orientation. The effects of label sparsity on the classification capacity of manifolds are elucidated, revealing a scaling relation between label sparsity and manifold radius. Theoretical predictions are corroborated by numerical simulations using recently developed algorithms to compute maximum margin solutions for manifold dichotomies. Our theory and its extensions provide a powerful and rich framework for applying statistical mechanics of linear classification to data arising from neuronal responses to object stimuli, as well as to artificial deep networks trained for object recognition tasks.

24 pages, 12 figures, Supplementary Materials

References in corpus (3)

• Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
• Linear Readout of Object Manifolds
• Learning Data Manifolds with a Cutting Plane Method

Cited by in corpus (42)

• Machine learning and the physical sciences
• Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
• Neural population geometry: An approach for understanding biological and artificial neural networks
• Intrinsic dimension of data representations in deep neural networks
• Modelling the influence of data structure on learning in neural networks: the hidden manifold model
• Perspectives on adaptive dynamical systems
• Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization
• Dimension of activity in random neural networks
• The Gaussian equivalence of generative models for learning with shallow neural networks
• Effective learning is accompanied by high dimensional and efficient representations of neural activity
• Dimensionality compression and expansion in Deep Neural Networks
• Mean-field inference methods for neural networks
• Mapping of attention mechanisms to a generalized Potts model
• Statistical learning theory of structured data
• Capacity-resolution trade-off in the optimal learning of multiple low-dimensional manifolds by attractor neural networks
• Counting the learnable functions of structured data
• Beyond the storage capacity: data driven satisfiability transition
• Linear Classification of Neural Manifolds with Correlated Variability
• On the geometry of generalization and memorization in deep neural networks
• Emergence of Separable Manifolds in Deep Language Representations
• Learning Data Manifolds with a Cutting Plane Method
• Generalization from correlated sets of patterns in the perceptron
• Random features and polynomial rules
• Neural tuning and representational geometry
• Capacity of the covariance perceptron
• Energy-information trade-off makes the cortical critical power law the optimal coding
• Statistical Mechanics of Support Vector Regression
• XMD: An Expansive Hardware-telemetry based Mobile Malware Detector to enhance Endpoint Detection
• Optimal Learning with Excitatory and Inhibitory synapses
• A new role for circuit expansion for learning in neural networks
• The impact of memory on learning sequence-to-sequence tasks
• Soft-margin classification of object manifolds
• Interpreting Encoding and Decoding Models
• Biologically inspired architectures for sample-efficient deep reinforcement learning
• Simplified derivations for high-dimensional convex learning problems
• Understanding the Logit Distributions of Adversarially-Trained Deep Neural Networks
• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?
• Dynamical Mechanism of Sampling-based Stochastic Inference under Probabilistic Population Codes
• Critical properties of the SAT/UNSAT transitions in the classification problem of structured data
• Optimal generalisation and learning transition in extensive-width shallow neural networks near interpolation
• Statistical mechanics of extensive-width Bayesian neural networks near interpolation
• Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation