Deep Learning: A Critical Appraisal

Abstract

Although deep learning has historical roots going back decades, neither the term "deep learning" nor the approach was popular just over five years ago, when the field was reignited by papers such as Krizhevsky, Sutskever and Hinton's now classic (2012) deep network model of Imagenet. What has the field discovered in the five subsequent years? Against a background of considerable progress in areas such as speech recognition, image recognition, and game playing, and considerable enthusiasm in the popular press, I present ten concerns for deep learning, and suggest that deep learning must be supplemented by other techniques if we are to reach artificial general intelligence.

1 figure

References in corpus (7)

• Deep Learning in Neural Networks: An Overview
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Adversarial Examples for Evaluating Reading Comprehension Systems
• Neural-Symbolic Learning and Reasoning: A Survey and Interpretation
• Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
• Frequently Asked Questions for: The Atoms of Neural Computation
• Visual Interaction Networks

Cited by in corpus (132)

• Shortcut Learning in Deep Neural Networks
• Deep Learning in Alzheimer's disease: Diagnostic Classification and Prognostic Prediction using Neuroimaging Data
• Human Activity Recognition using Inertial, Physiological and Environmental Sensors: a Comprehensive Survey
• Invariant Risk Minimization
• The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
• Autonomous discovery in the chemical sciences part II: Outlook
• Limits of trust in medical AI
• Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
• FinBERT: Financial Sentiment Analysis with Pre-trained Language Models
• Hybrid FEM-NN models: Combining artificial neural networks with the finite element method
• Scalable agent alignment via reward modeling: a research direction
• Recent Advances in Deep Learning: An Overview
• Assessing Generalization in Deep Reinforcement Learning
• An Ensemble of Convolutional Neural Networks for Audio Classification
• Deep matrix factorizations
• Innateness, AlphaZero, and Artificial Intelligence
• Compositional generalization in a deep seq2seq model by separating syntax and semantics
• ProCAN: Progressive Growing Channel Attentive Non-Local Network for Lung Nodule Classification
• Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs
• Decay Replay Mining to Predict Next Process Events
• A Boxology of Design Patterns for Hybrid Learning and Reasoning Systems
• A Hierarchy of Limitations in Machine Learning
• Out of Distribution Generalization in Machine Learning
• Compositional generalization through meta sequence-to-sequence learning
• On Adversarial Examples and Stealth Attacks in Artificial Intelligence Systems
• Unsupervised Model Selection for Variational Disentangled Representation Learning
• Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
• Known Operator Learning and Hybrid Machine Learning in Medical Imaging -- A Review of the Past, the Present, and the Future
• Learning Compositional Rules via Neural Program Synthesis
• Towards Safe Machine Learning for CPS: Infer Uncertainty from Training Data
• Cross-Domain Few-Shot Learning by Representation Fusion
• An Explicitly Relational Neural Network Architecture
• Opening the black box of deep learning
• Deep Stable Learning for Out-Of-Distribution Generalization
• Deep Nets: What have they ever done for Vision?
• Towards Neural Theorem Proving at Scale
• Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization
• Robust Conversational AI with Grounded Text Generation
• Transforming task representations to perform novel tasks
• Analyzing Differentiable Fuzzy Logic Operators
• Don't Explain without Verifying Veracity: An Evaluation of Explainable AI with Video Activity Recognition
• Cyberbullying Detection -- Technical Report 2/2018, Department of Computer Science AGH, University of Science and Technology
• Artificial intelligence empowered multi-AGVs in manufacturing systems
• Synergetic Learning Systems: Concept, Architecture, and Algorithms
• Differentiable Logic Machines
• "Weak AI" is Likely to Never Become "Strong AI", So What is its Greatest Value for us?
• Learning Online Visual Invariances for Novel Objects via Supervised and Self-Supervised Training
• ML-PersRef: A Machine Learning-based Personalized Multimodal Fusion Approach for Referencing Outside Objects From a Moving Vehicle
• I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling
• Dimension Estimation Using Autoencoders
• Hybrid quantum-classical unsupervised data clustering based on the self-organizing feature map
• Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning
• Scaling Down Deep Learning with MNIST-1D
• Solutions to problems with deep learning
• Visual Representation Learning Does Not Generalize Strongly Within the Same Domain
• A New Approach for Explainable Multiple Organ Annotation with Few Data
• Deep Adaptive Semantic Logic (DASL): Compiling Declarative Knowledge into Deep Neural Networks
• B-HAR: an open-source baseline framework for in depth study of human activity recognition datasets and workflows
• Lexicon Learning for Few-Shot Neural Sequence Modeling
• Logic Programming and Machine Ethics
• JUMPER: Learning When to Make Classification Decisions in Reading
• An Approach Towards Physics Informed Lung Ultrasound Image Scoring Neural Network for Diagnostic Assistance in COVID-19
• Data-Centric Engineering: integrating simulation, machine learning and statistics. Challenges and Opportunities
• The 30-Year Cycle In The AI Debate
• Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning
• Levels of Analysis for Machine Learning
• Giving Up Control: Neurons as Reinforcement Learning Agents
• Data Curation with Deep Learning [Vision]
• Symbolic Behaviour in Artificial Intelligence
• Pyramid Fusion Dark Channel Prior for Single Image Dehazing
• Automated cardiovascular magnetic resonance image analysis with fully convolutional networks
• Learning Contextualized Knowledge Structures for Commonsense Reasoning
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study
• A model of cortical cognitive function using hierarchical interactions of gating matrices in internal agents coding relational representations
• ECG-DelNet: Delineation of Ambulatory Electrocardiograms with Mixed Quality Labeling Using Neural Networks
• Weight Priors for Learning Identity Relations
• The Privatization of AI Research(-ers): Causes and Potential Consequences -- From university-industry interaction to public research brain-drain?
• Text Classification for Azerbaijani Language Using Machine Learning and Embedding
• Learning Translation Invariance in CNNs
• A decision theoretic approach to model evaluation in computational drug discovery
• The relational processing limits of classic and contemporary neural network models of language processing
• Distinguishing rule- and exemplar-based generalization in learning systems
• Visual-Semantic Embedding Model Informed by Structured Knowledge
• On the Ethics of Building AI in a Responsible Manner
• Filter Methods for Feature Selection in Supervised Machine Learning Applications -- Review and Benchmark
• On (Emergent) Systematic Generalisation and Compositionality in Visual Referential Games with Straight-Through Gumbel-Softmax Estimator
• Diagnosing the performance of human mobility models at small spatial scales using volunteered geographic information
• Shallow Unorganized Neural Networks using Smart Neuron Model for Visual Perception
• Roadmap to a Roadmap: How Could We Tell When AGI is a 'Manhattan Project' Away?
• A Road Map to Strong Intelligence
• Neural Abstract Reasoner
• Learning a metacognition for object perception
• Untangling urban data signatures: unsupervised machine learning methods for the detection of urban archetypes at the pedestrian scale
• SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning
• Taking Principles Seriously: A Hybrid Approach to Value Alignment
• Learning model-based strategies in simple environments with hierarchical q-networks
• Ontology and Cognitive Outcomes
• The curious case of developmental BERTology: On sparsity, transfer learning, generalization and the brain
• Questions to Guide the Future of Artificial Intelligence Research
• Deep Learning in Science
• $p{i}_t$- Enhancing the Precision of Eye Tracking using Iris Feature Motion Vectors
• Human-Like Decision Making: Document-level Aspect Sentiment Classification via Hierarchical Reinforcement Learning
• Zero-shot task adaptation by homoiconic meta-mapping
• Deep learning for image segmentation: veritable or overhyped?
• A New Constructive Heuristic driven by Machine Learning for the Traveling Salesman Problem
• Modular meta-learning in abstract graph networks for combinatorial generalization
• Simulating extrapolated dynamics with parameterization networks
• Knowledge-driven Active Learning
• Radically Compositional Cognitive Concepts
• A New Framework for Machine Intelligence: Concepts and Prototype
• Relational Weight Priors in Neural Networks for Abstract Pattern Learning and Language Modelling
• Unsupervised Spiking Instance Segmentation on Event Data using STDP
• Towards sample-efficient episodic control with DAC-ML
• Unsupervised Deep Learning by Injecting Low-Rank and Sparse Priors
• Modelling Identity Rules with Neural Networks
• A neural network model of perception and reasoning
• Toward Building Science Discovery Machines
• Prediction of 'artificial' urban archetypes at the pedestrian-scale through a synthesis of domain expertise with machine learning methods
• Predicate learning in neural systems: Discovering latent generative structures
• Fit to Measure: Reasoning about Sizes for Robust Object Recognition
• Towards Efficient and Secure Delivery of Data for Training and Inference with Privacy-Preserving
• Towards Model-informed Precision Dosing with Expert-in-the-loop Machine Learning
• Dual approach for object tracking based on optical flow and swarm intelligence
• Computational principles of intelligence: learning and reasoning with neural networks
• Prevention is Better than Cure: Handling Basis Collapse and Transparency in Dense Networks
• Towards Efficient and Secure Delivery of Data for Deep Learning with Privacy-Preserving
• Code Farming: A Process for Creating Generic Computational Building Blocks
• blessing in disguise: Designing Robust Turing Test by Employing Algorithm Unrobustness
• Formal Fields: A Framework to Automate Code Generation Across Domains
• Training Deep Normalizing Flow Models in Highly Incomplete Data Scenarios with Prior Regularization
• Fast Jacobian-Vector Product for Deep Networks
• Robust Cell-Load Learning with a Small Sample Set