A Fast Incremental Gaussian Mixture Model

Abstract

This work builds upon previous efforts in online incremental learning, namely the Incremental Gaussian Mixture Network (IGMN). The IGMN is capable of learning from data streams in a single-pass by improving its model after analyzing each data point and discarding it thereafter. Nevertheless, it suffers from the scalability point-of-view, due to its asymptotic time complexity of $O\left(NK{D}^3\right)$ for $N$ data points, $K$ Gaussian components and $D$ dimensions, rendering it inadequate for high-dimensional data. In this paper, we manage to reduce this complexity to $O\left(NK{D}^2\right)$ by deriving formulas for working directly with precision matrices instead of covariance matrices. The final result is a much faster and scalable algorithm which can be applied to high dimensional tasks. This is confirmed by applying the modified algorithm to high-dimensional classification datasets.

10 pages, no figures, draft submission to Plos One

References in corpus (2)

• Improving neural networks by preventing co-adaptation of feature detectors
• A Fast Incremental Gaussian Mixture Model

Cited by in corpus (9)

• A Fast Incremental Gaussian Mixture Model
• On the detection-to-track association for online multi-object tracking
• Accurate Molecular-Orbital-Based Machine Learning Energies via Unsupervised Clustering of Chemical Space
• Hierarchical Quantized Autoencoders
• Scalable and Incremental Learning of Gaussian Mixture Models
• Physically-inspired computational tools for sharp detection of material inhomogeneities in magnetic imaging
• Online Unsupervised Learning of Visual Representations and Categories
• An empirical learning-based validation procedure for simulation workflow
• Continual Competitive Memory: A Neural System for Online Task-Free Lifelong Learning