Role of Delay in Brain Dynamics

Abstract

Significant variations of delays among connecting neurons cause an inevitable disadvantage of asynchronous brain dynamics compared to synchronous deep learning. However, this study demonstrates that this disadvantage can be converted into a computational advantage using a network with a single output and M multiple delays between successive layers, thereby generating a polynomial time-series outputs with M. The proposed role of delay in brain dynamics (RoDiB) model, is capable of learning increasing number of classified labels using a fixed architecture, and overcomes the inflexibility of the brain to update the learning architecture using additional neurons and connections. Moreover, the achievable accuracies of the RoDiB system are comparable with those of its counterpart tunable single delay architectures with M outputs. Further, the accuracies are significantly enhanced when the number of output labels exceeds its fully connected input size. The results are mainly obtained using simulations of VGG-6 on CIFAR datasets and also include multiple label inputs. However, currently only a small fraction of the abundant number of RoDiB outputs is utilized, thereby suggesting its potential for advanced computational power yet to be discovered.

18 pages, 3 figures, 2 tables

References in corpus (14)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Learning in Neural Networks: An Overview
• Phase transitions in Restricted Boltzmann Machines with generic priors
• Computational paradigm for dynamic logic-gates in neuronal activity
• Efficient shallow learning as an alternative to deep learning
• Power-law Scaling to Assist with Key Challenges in Artificial Intelligence
• Neuronal Response Impedance Mechanism Implementing Cooperative Networks with Low Firing Rates and Microseconds Precision
• Towards a universal mechanism for successful deep learning
• Phase transitions in the mini-batch size for sparse and dense two-layer neural networks
• Long anisotropic absolute refractory periods with rapid rise-times to reliable responsiveness
• Learning on tree architectures outperforms a convolutional feedforward network
• Significant anisotropic neuronal refractory period plasticity
• Brain inspired neuronal silencing mechanism to enable reliable sequence identification
• Enhancing the accuracies by performing pooling decisions adjacent to the output layer

Cited by in corpus (1)

• Unified CNNs and transformers underlying learning mechanism reveals multi-head attention modus vivendi