A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations

Abstract

This two-part comprehensive survey is devoted to a computing framework most commonly known under the names Hyperdimensional Computing and Vector Symbolic Architectures (HDC/VSA). Both names refer to a family of computational models that use high-dimensional distributed representations and rely on the algebraic properties of their key operations to incorporate the advantages of structured symbolic representations and vector distributed representations. Notable models in the HDC/VSA family are Tensor Product Representations, Holographic Reduced Representations, Multiply-Add-Permute, Binary Spatter Codes, and Sparse Binary Distributed Representations but there are other models too. HDC/VSA is a highly interdisciplinary field with connections to computer science, electrical engineering, artificial intelligence, mathematics, and cognitive science. This fact makes it challenging to create a thorough overview of the field. However, due to a surge of new researchers joining the field in recent years, the necessity for a comprehensive survey of the field has become extremely important. Therefore, amongst other aspects of the field, this Part I surveys important aspects such as: known computational models of HDC/VSA and transformations of various input data types to high-dimensional distributed representations. Part II of this survey is devoted to applications, cognitive computing and architectures, as well as directions for future work. The survey is written to be useful for both newcomers and practitioners.

31 pages

References in corpus (16)

• Classification using Hyperdimensional Computing: A Review
• A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges
• Robust High-dimensional Memory-augmented Neural Networks
• A comparison of Vector Symbolic Architectures
• A Theoretical Perspective on Hyperdimensional Computing
• Neural Distributed Autoassociative Memories: A Survey
• Tensor-product vs. geometric-product coding
• Understanding Hyperdimensional Computing for Parallel Single-Pass Learning
• Efficient Decoding of Compositional Structure in Holistic Representations
• On Geometric Algebra representation of Binary Spatter Codes
• Shift-Equivariant Similarity-Preserving Hypervector Representations of Sequences
• Generalized Key-Value Memory to Flexibly Adjust Redundancy in Memory-Augmented Networks
• Computing on Functions Using Randomized Vector Representations
• Analyzing the Capacity of Distributed Vector Representations to Encode Spatial Information
• A novel HD Computing Algebra: Non-associative superposition of states creating sparse bundles representing order information
• Optimal quadratic binding for relational reasoning in vector symbolic neural architectures

Cited by in corpus (19)

• Vector Symbolic Architectures as a Computing Framework for Emerging Hardware
• Perceptron Theory Can Predict the Accuracy of Neural Networks
• Neuromorphic Visual Scene Understanding with Resonator Networks
• Training a HyperDimensional Computing Classifier using a Threshold on its Confidence
• An Encoding Framework for Binarized Images using HyperDimensional Computing
• Efficient Decoding of Compositional Structure in Holistic Representations
• Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy
• Vector Symbolic Finite State Machines in Attractor Neural Networks
• Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices
• Generalized Key-Value Memory to Flexibly Adjust Redundancy in Memory-Augmented Networks
• Optical hyperdimensional soft sensing: Speckle-based touch interface and tactile sensor
• WHYPE: A Scale-Out Architecture with Wireless Over-the-Air Majority for Scalable In-memory Hyperdimensional Computing
• Hyperdimensional Representation Learning for Node Classification and Link Prediction
• Distributed Representations Enable Robust Multi-Timescale Symbolic Computation in Neuromorphic Hardware
• Compositional Factorization of Visual Scenes with Convolutional Sparse Coding and Resonator Networks
• On Design Choices in Similarity-Preserving Sparse Randomized Embeddings
• Hyperdimensional Decoding of Spiking Neural Networks
• FactorHD: A Hyperdimensional Computing Model for Multi-Object Multi-Class Representation and Factorization
• Multiclass Linear Perceptrons with Multiplicative Margins