High--Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

Abstract

High-dimensional data and high-dimensional representations of reality are inherent features of modern Artificial Intelligence systems and applications of machine learning. The well-known phenomenon of the "curse of dimensionality" states: many problems become exponentially difficult in high dimensions. Recently, the other side of the coin, the "blessing of dimensionality", has attracted much attention. It turns out that generic high-dimensional datasets exhibit fairly simple geometric properties. Thus, there is a fundamental tradeoff between complexity and simplicity in high dimensional spaces. Here we present a brief explanatory review of recent ideas, results and hypotheses about the blessing of dimensionality and related simplifying effects relevant to machine learning and neuroscience.

arXiv comment18 pages, 5 figures

References in corpus (14)

• Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1
• Observed Universality of Phase Transitions in High-Dimensional Geometry, with Implications for Modern Data Analysis and Signal Processing
• Principal manifolds and graphs in practice: from molecular biology to dynamical systems
• Approximation with Random Bases: Pro et Contra
• Blessing of dimensionality: mathematical foundations of the statistical physics of data
• Correction of AI systems by linear discriminants: Probabilistic foundations
• The unreasonable effectiveness of small neural ensembles in high-dimensional brain
• Stochastic Separation Theorems
• The Remarkable Simplicity of Very High Dimensional Data: Application of Model-Based Clustering
• High-dimensional brain. A tool for encoding and rapid learning of memories by single neurons
• One-Trial Correction of Legacy AI Systems and Stochastic Separation Theorems
• The More, the Merrier: the Blessing of Dimensionality for Learning Large Gaussian Mixtures
• Fast Construction of Correcting Ensembles for Legacy Artificial Intelligence Systems: Algorithms and a Case Study
• Symphony of high-dimensional brain

Cited by in corpus (4)

• Fractional norms and quasinorms do not help to overcome the curse of dimensionality
• High-dimensional separability for one- and few-shot learning
• Blessing of dimensionality at the edge
• Linear and Fisher Separability of Random Points in the d-dimensional Spherical Layer