Properties of Sparse Distributed Representations and their Application to Hierarchical Temporal Memory

Abstract

Empirical evidence demonstrates that every region of the neocortex represents information using sparse activity patterns. This paper examines Sparse Distributed Representations (SDRs), the primary information representation strategy in Hierarchical Temporal Memory (HTM) systems and the neocortex. We derive a number of properties that are core to scaling, robustness, and generalization. We use the theory to provide practical guidelines and illustrate the power of SDRs as the basis of HTM. Our goal is to help create a unified mathematical and practical framework for SDRs as it relates to cortical function.

Cited by in corpus (5)

• Using Spatial Pooler of Hierarchical Temporal Memory to classify noisy videos with predefined complexity
• Non-volatile Hierarchical Temporal Memory: Hardware for Spatial Pooling
• Superposed Episodic and Semantic Memory via Sparse Distributed Representation
• The observer-assisted method for adjusting hyper-parameters in deep learning algorithms
• Deviant Learning Algorithm: Learning Sparse Mismatch Representations through Time and Space