Properties of Sparse Distributed Representations and their Application to Hierarchical Temporal Memory
arXiv:1503.07469
Abstract
Empirical evidence demonstrates that every region of the neocortex represents information using sparse activity patterns. This paper examines Sparse Distributed Representations (SDRs), the primary information representation strategy in Hierarchical Temporal Memory (HTM) systems and the neocortex. We derive a number of properties that are core to scaling, robustness, and generalization. We use the theory to provide practical guidelines and illustrate the power of SDRs as the basis of HTM. Our goal is to help create a unified mathematical and practical framework for SDRs as it relates to cortical function.
Cited by in corpus (20)
- Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex
- Continuous online sequence learning with an unsupervised neural network model
- A comparison of Vector Symbolic Architectures
- Sequence learning, prediction, and replay in networks of spiking neurons
- End-to-End Memristive HTM System for Pattern Recognition and Sequence Prediction
- Porting HTM Models to the Heidelberg Neuromorphic Computing Platform
- Using Spatial Pooler of Hierarchical Temporal Memory to classify noisy videos with predefined complexity
- Non-volatile Hierarchical Temporal Memory: Hardware for Spatial Pooling
- Encoding Reality: Prediction-Assisted Cortical Learning Algorithm in Hierarchical Temporal Memory
- The Utility of Sparse Representations for Control in Reinforcement Learning
- CLAASIC: a Cortex-Inspired Hardware Accelerator
- Superposed Episodic and Semantic Memory via Sparse Distributed Representation
- Impact of Driving Behavior on Commuter's Comfort during Cab Rides: Towards a New Perspective of Driver Rating
- Symphony from Synapses: Neocortex as a Universal Dynamical Systems Modeller using Hierarchical Temporal Memory
- The observer-assisted method for adjusting hyper-parameters in deep learning algorithms
- Deviant Learning Algorithm: Learning Sparse Mismatch Representations through Time and Space
- A Road Map to Strong Intelligence
- Exponential scaling of neural algorithms - a future beyond Moore's Law?
- Towards Machine Intelligence
- Ultra-High Dimensional Sparse Representations with Binarization for Efficient Text Retrieval