Generalized Learning Vector Quantization for Classification in Randomized Neural Networks and Hyperdimensional Computing

Abstract

Machine learning algorithms deployed on edge devices must meet certain resource constraints and efficiency requirements. Random Vector Functional Link (RVFL) networks are favored for such applications due to their simple design and training efficiency. We propose a modified RVFL network that avoids computationally expensive matrix operations during training, thus expanding the network's range of potential applications. Our modification replaces the least-squares classifier with the Generalized Learning Vector Quantization (GLVQ) classifier, which only employs simple vector and distance calculations. The GLVQ classifier can also be considered an improvement upon certain classification algorithms popularly used in the area of Hyperdimensional Computing. The proposed approach achieved state-of-the-art accuracy on a collection of datasets from the UCI Machine Learning Repository - higher than previously proposed RVFL networks. We further demonstrate that our approach still achieves high accuracy while severely limited in training iterations (using on average only 21% of the least-squares classifier computational costs).

arXiv comment10 pages, 7 figures

References in corpus (8)

• Classification using Hyperdimensional Computing: A Review
• Benchmarking TinyML Systems: Challenges and Direction
• Vector Symbolic Architectures answer Jackendoff's challenges for cognitive neuroscience
• A review of learning vector quantization classifiers
• Density Encoding Enables Resource-Efficient Randomly Connected Neural Networks
• Random Vector Functional Link Networks for Function Approximation on Manifolds
• Autoscaling Bloom Filter: Controlling Trade-off Between True and False Positives
• HyperEmbed: Tradeoffs Between Resources and Performance in NLP Tasks with Hyperdimensional Computing enabled Embedding of n-gram Statistics