Understanding in Artificial Intelligence

Abstract

Current Artificial Intelligence (AI) methods, most based on deep learning, have facilitated progress in several fields, including computer vision and natural language understanding. The progress of these AI methods is measured using benchmarks designed to solve challenging tasks, such as visual question answering. A question remains of how much understanding is leveraged by these methods and how appropriate are the current benchmarks to measure understanding capabilities. To answer these questions, we have analysed existing benchmarks and their understanding capabilities, defined by a set of understanding capabilities, and current research streams. We show how progress has been made in benchmark development to measure understanding capabilities of AI methods and we review as well how current methods develop understanding capabilities.

28 pages, 282 references

References in corpus (80)

• Adam: A Method for Stochastic Optimization
• Deep Learning in Neural Networks: An Overview
• Distributed Representations of Words and Phrases and their Compositionality
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Playing Atari with Deep Reinforcement Learning
• Cooperative Game Theory Approaches for Network Partitioning
• Language Models are Few-Shot Learners
• Teaching Machines to Read and Comprehend
• A Review of Relational Machine Learning for Knowledge Graphs
• Unsupervised Learning of Video Representations using LSTMs
• Microsoft COCO Captions: Data Collection and Evaluation Server
• Tutorial on Variational Autoencoders
• Hierarchical Question-Image Co-Attention for Visual Question Answering
• DRAW: A Recurrent Neural Network For Image Generation
• ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• FCOS: Fully Convolutional One-Stage Object Detection
• Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
• Gated Graph Sequence Neural Networks
• CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
• The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
• Video (language) modeling: a baseline for generative models of natural videos
• Visual Question Answering: Datasets, Algorithms, and Future Challenges
• A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input
• Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
• Multi-Object Representation Learning with Iterative Variational Inference
• Neural Programmer-Interpreters
• LXMERT: Learning Cross-Modality Encoder Representations from Transformers
• Towards Automatic Learning of Procedures from Web Instructional Videos
• ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension
• Harnessing Deep Neural Networks with Logic Rules
• MONet: Unsupervised Scene Decomposition and Representation
• Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks
• Measuring abstract reasoning in neural networks
• Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning
• Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
• A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories
• SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition
• Logical Neural Networks
• TerpreT: A Probabilistic Programming Language for Program Induction
• NewsQA: A Machine Comprehension Dataset
• CLEVRER: CoLlision Events for Video REpresentation and Reasoning
• Dynamic Integration of Background Knowledge in Neural NLU Systems
• Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches
• Neural Embeddings of Graphs in Hyperbolic Space
• BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation
• Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders
• A Wrapped Normal Distribution on Hyperbolic Space for Gradient-Based Learning
• C-VQA: A Compositional Split of the Visual Question Answering (VQA) v1.0 Dataset
• On Interpretability of Artificial Neural Networks: A Survey
• Hyperbolic Entailment Cones for Learning Hierarchical Embeddings
• Differentiable Perturb-and-Parse: Semi-Supervised Parsing with a Structured Variational Autoencoder
• KBGAN: Adversarial Learning for Knowledge Graph Embeddings
• SOLO: Segmenting Objects by Locations
• Image-based table recognition: data, model, and evaluation
• Automatic Program Synthesis of Long Programs with a Learned Garbage Collector
• Stochastic Answer Networks for Machine Reading Comprehension
• Differentiable Programs with Neural Libraries
• STAIR Actions: A Video Dataset of Everyday Home Actions
• Hyperbolic Disk Embeddings for Directed Acyclic Graphs
• Improving Question Answering over Incomplete KBs with Knowledge-Aware Reader
• Recent Advances in Neural Program Synthesis
• Deep Learning for Learning Graph Representations
• An Explicit Local and Global Representation Disentanglement Framework with Applications in Deep Clustering and Unsupervised Object Detection
• Commonsense for Generative Multi-Hop Question Answering Tasks
• Cracking the Contextual Commonsense Code: Understanding Commonsense Reasoning Aptitude of Deep Contextual Representations
• Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehension
• On the Possibilities and Limitations of Multi-hop Reasoning Under Linguistic Imperfections
• Answering Science Exam Questions Using Query Rewriting with Background Knowledge
• AMR-to-text Generation with Synchronous Node Replacement Grammar
• Answering Visual What-If Questions: From Actions to Predicted Scene Descriptions
• Can a Gorilla Ride a Camel? Learning Semantic Plausibility from Text
• Reasoning about Actions and State Changes by Injecting Commonsense Knowledge
• Ranking and Selecting Multi-Hop Knowledge Paths to Better Predict Human Needs
• A Neural-Symbolic Architecture for Inverse Graphics Improved by Lifelong Meta-Learning
• Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic Spaces
• Winograd Schema - Knowledge Extraction Using Narrative Chains