A crossover code for high-dimensional composition

Abstract

We present a novel way to encode compositional information in high-dimensional (HD) vectors. Inspired by chromosomal crossover, random HD vectors are recursively interwoven, with a fraction of one vector's components masked out and replaced by those from another using a context-dependent mask. Unlike many HD computing schemes, "crossover" codes highly overlap with their base elements' and sub-structures' codes without sacrificing relational information, allowing fast element readout and decoding by greedy reconstruction. Crossover is mathematically tractable and has several properties desirable for robust, flexible representation.

Presented at NeurIPS 2019 Workshop on Context and Compositionality in Biological and Artificial Neural Systems

References in corpus (2)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Methods for Interpreting and Understanding Deep Neural Networks