Correction of AI systems by linear discriminants: Probabilistic foundations

Abstract

Artificial Intelligence (AI) systems sometimes make errors and will make errors in the future, from time to time. These errors are usually unexpected, and can lead to dramatic consequences. Intensive development of AI and its practical applications makes the problem of errors more important. Total re-engineering of the systems can create new errors and is not always possible due to the resources involved. The important challenge is to develop fast methods to correct errors without damaging existing skills. We formulated the technical requirements to the 'ideal' correctors. Such correctors include binary classifiers, which separate the situations with high risk of errors from the situations where the AI systems work properly. Surprisingly, for essentially high-dimensional data such methods are possible: simple linear Fisher discriminant can separate the situations with errors from correctly solved tasks even for exponentially large samples. The paper presents the probabilistic basis for fast non-destructive correction of AI systems. A series of new stochastic separation theorems is proven. These theorems provide new instruments for fast non-iterative correction of errors of legacy AI systems. The new approaches become efficient in high-dimensions, for correction of high-dimensional systems in high-dimensional world (i.e. for processing of essentially high-dimensional data by large systems).

arXiv admin note: text overlap with arXiv:1809.07656 and arXiv:1802.02172

References in corpus (13)

• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Stochastic Configuration Networks: Fundamentals and Algorithms
• Observed Universality of Phase Transitions in High-Dimensional Geometry, with Implications for Modern Data Analysis and Signal Processing
• To Trust Or Not To Trust A Classifier
• Approximation with Random Bases: Pro et Contra
• Blessing of dimensionality: mathematical foundations of the statistical physics of data
• Neurogenesis Deep Learning
• Stochastic Separation Theorems
• Knowledge Transfer Between Artificial Intelligence Systems
• One-Trial Correction of Legacy AI Systems and Stochastic Separation Theorems
• The More, the Merrier: the Blessing of Dimensionality for Learning Large Gaussian Mixtures
• Augmented Artificial Intelligence: a Conceptual Framework
• How deep should be the depth of convolutional neural networks: a backyard dog case study

Cited by in corpus (19)

• Scikit-dimension: a Python package for intrinsic dimension estimation
• The unreasonable effectiveness of small neural ensembles in high-dimensional brain
• Dynamic and Thermodynamic Models of Adaptation
• High--Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality
• Fractional norms and quasinorms do not help to overcome the curse of dimensionality
• On Adversarial Examples and Stealth Attacks in Artificial Intelligence Systems
• Knowledge Transfer Between Artificial Intelligence Systems
• General stochastic separation theorems with optimal bounds
• Fast Construction of Correcting Ensembles for Legacy Artificial Intelligence Systems: Algorithms and a Case Study
• How adversarial attacks can disrupt seemingly stable accurate classifiers
• High-dimensional separability for one- and few-shot learning
• The Feasibility and Inevitability of Stealth Attacks
• Blessing of dimensionality at the edge
• Local intrinsic dimensionality estimators based on concentration of measure
• MyI-Net: Fully Automatic Detection and Quantification of Myocardial Infarction from Cardiovascular MRI Images
• Symphony of high-dimensional brain
• Linear and Fisher Separability of Random Points in the d-dimensional Spherical Layer
• Machine Learning approach to the Floquet--Lindbladian problem
• Relative intrinsic dimensionality is intrinsic to learning