BEAN: Interpretable Representation Learning with Biologically-Enhanced Artificial Neuronal Assembly Regularization

Abstract

Deep neural networks (DNNs) are known for extracting useful information from large amounts of data. However, the representations learned in DNNs are typically hard to interpret, especially in dense layers. One crucial issue of the classical DNN model such as multilayer perceptron (MLP) is that neurons in the same layer of DNNs are conditionally independent of each other, which makes co-training and emergence of higher modularity difficult. In contrast to DNNs, biological neurons in mammalian brains display substantial dependency patterns. Specifically, biological neural networks encode representations by so-called neuronal assemblies: groups of neurons interconnected by strong synaptic interactions and sharing joint semantic content. The resulting population coding is essential for human cognitive and mnemonic processes. Here, we propose a novel Biologically Enhanced Artificial Neuronal assembly (BEAN) regularization to model neuronal correlations and dependencies, inspired by cell assembly theory from neuroscience. Experimental results show that BEAN enables the formation of interpretable neuronal functional clusters and consequently promotes a sparse, memory/computation-efficient network without loss of model performance. Moreover, our few-shot learning experiments demonstrate that BEAN could also enhance the generalizability of the model when training samples are extremely limited.

Accepted as an original research paper at Frontiers in Neurorobotics

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Prototypical Networks for Few-shot Learning
• Recent Trends in Deep Learning Based Natural Language Processing
• Zero-Shot Learning Through Cross-Modal Transfer
• Group Sparse Regularization for Deep Neural Networks
• On the importance of single directions for generalization
• Revisiting the Importance of Individual Units in CNNs via Ablation