Artificial neural networks for neuroscientists: A primer

Abstract

Artificial neural networks (ANNs) are essential tools in machine learning that have drawn increasing attention in neuroscience. Besides offering powerful techniques for data analysis, ANNs provide a new approach for neuroscientists to build models for complex behaviors, heterogeneous neural activity and circuit connectivity, as well as to explore optimization in neural systems, in ways that traditional models are not designed for. In this pedagogical Primer, we introduce ANNs and demonstrate how they have been fruitfully deployed to study neuroscientific questions. We first discuss basic concepts and methods of ANNs. Then, with a focus on bringing this mathematical framework closer to neurobiology, we detail how to customize the analysis, structure, and learning of ANNs to better address a wide range of challenges in brain research. To help the readers garner hands-on experience, this Primer is accompanied with tutorial-style code in PyTorch and Jupyter Notebook, covering major topics.

References in corpus (42)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Auto-Encoding Variational Bayes
• Neural Machine Translation by Jointly Learning to Align and Translate
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• TensorFlow: A system for large-scale machine learning
• Overcoming catastrophic forgetting in neural networks
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Intriguing properties of neural networks
• On the difficulty of training Recurrent Neural Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Deep Learning in Spiking Neural Networks
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Continual Learning Through Synaptic Intelligence
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
• Opening the Black Box of Deep Neural Networks via Information
• Demixed principal component analysis of population activity in higher cortical areas reveals independent representation of task parameters
• A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
• Complexity without chaos: Plasticity within random recurrent networks generates robust timing and motor control
• SuperSpike: Supervised learning in multi-layer spiking neural networks
• Layer Normalization
• Recurrence is required to capture the representational dynamics of the human visual system
• Linking connectivity, dynamics and computations in low-rank recurrent neural networks
• Similarity of Neural Network Representations Revisited
• Recurrent Network Models Of Sequence Generation And Memory
• Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
• Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
• A mathematical theory of semantic development in deep neural networks
• Supervised Learning in Spiking Neural Networks with FORCE Training
• Emergence of grid-like representations by training recurrent neural networks to perform spatial localization
• Deep Learning Models of the Retinal Response to Natural Scenes
• Using Fast Weights to Attend to the Recent Past
• Task-Driven Convolutional Recurrent Models of the Visual System
• Meta-Learning Update Rules for Unsupervised Representation Learning
• Universality and individuality in neural dynamics across large populations of recurrent networks
• Dendritic error backpropagation in deep cortical microcircuits

Cited by in corpus (4)

• Measuring and modeling the motor system with machine learning
• Reservoir Computing and Photoelectrochemical Sensors: A Marriage of Convenience
• Solving large-scale MEG/EEG source localization and functional connectivity problems simultaneously using state-space models
• Dynamical Mechanism of Sampling-based Stochastic Inference under Probabilistic Population Codes