Bridge Networks: Relating Inputs through Vector-Symbolic Manipulations

Abstract

Despite rapid progress, current deep learning methods face a number of critical challenges. These include high energy consumption, catastrophic forgetting, dependance on global losses, and an inability to reason symbolically. By combining concepts from information bottleneck theory and vector-symbolic architectures, we propose and implement a novel information processing architecture, the 'Bridge network.' We show this architecture provides unique advantages which can address the problem of global losses and catastrophic forgetting. Furthermore, we argue that it provides a further basis for increasing energy efficiency of execution and the ability to reason symbolically.

arXiv comment6 pages, 6 figures

References in corpus (7)

• A Survey of Neuromorphic Computing and Neural Networks in Hardware
• Measuring Catastrophic Forgetting in Neural Networks
• A comparison of Vector Symbolic Architectures
• Soft-Label Dataset Distillation and Text Dataset Distillation
• Neuromorphic Hardware learns to learn
• Dataset Distillation
• Resonator networks for factoring distributed representations of data structures