Understanding Attention: In Minds and Machines
arXiv:2012.02659
Abstract
Attention is a complex and broad concept, studied across multiple disciplines spanning artificial intelligence, cognitive science, psychology, neuroscience, and related fields. Although many of the ideas regarding attention do not significantly overlap among these fields, there is a common theme of adaptive control of limited resources. In this work, we review the concept and variants of attention in artificial neural networks (ANNs). We also discuss the origin of attention from the neuroscience point of view parallel to that of ANNs. Instead of having seemingly disconnected dialogues between varied disciplines, we suggest grounding the ideas on common conceptual frameworks for a systematic analysis of attention and towards possible unification of ideas in AI and Neuroscience.
arXiv commentAccepted at NeurIPS 2020 Workshop: ML Retrospectives, Surveys & Meta-Analyses (ML-RSA)
References in corpus (5)
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
- A Survey of Neuromorphic Computing and Neural Networks in Hardware
- Towards a Definition of Disentangled Representations
- Levels of Analysis for Machine Learning