Emergence of Concepts in DNNs?
arXiv:2211.06137 · doi:10.1016/j.patter.2023.100761
Abstract
The present paper reviews and discusses work from computer science that proposes to identify concepts in internal representations (hidden layers) of DNNs. It is examined, first, how existing methods actually identify concepts that are supposedly represented in DNNs. Second, it is discussed how conceptual spaces -- sets of concepts in internal representations -- are shaped by a tradeoff between predictive accuracy and compression. These issues are critically examined by drawing on philosophy. While there is evidence that DNNs able to represent non-trivial inferential relations between concepts, our ability to identify concepts is severely limited.
arXiv comment13 pages, 2 figures
References in corpus (6)
- Reconciling modern machine learning practice and the bias-variance trade-off
- Understanding the Role of Individual Units in a Deep Neural Network
- Concept Whitening for Interpretable Image Recognition
- The Modern Mathematics of Deep Learning
- Achieving Conservation of Energy in Neural Network Emulators for Climate Modeling
- Efficient human-like semantic representations via the Information Bottleneck principle