A Hybrid Neural Network Model for Commonsense Reasoning
arXiv:1907.11983
Abstract
This paper proposes a hybrid neural network (HNN) model for commonsense reasoning. An HNN consists of two component models, a masked language model and a semantic similarity model, which share a BERT-based contextual encoder but use different model-specific input and output layers. HNN obtains new state-of-the-art results on three classic commonsense reasoning tasks, pushing the WNLI benchmark to 89%, the Winograd Schema Challenge (WSC) benchmark to 75.1%, and the PDP60 benchmark to 90.0%. An ablation study shows that language models and semantic similarity models are complementary approaches to commonsense reasoning, and HNN effectively combines the strengths of both. The code and pre-trained models will be publicly available at https://github.com/namisan/mt-dnn.
9 pages, 3 figures, 6 tables
References in corpus (7)
- A Simple Method for Commonsense Reasoning
- Averaging Weights Leads to Wider Optima and Better Generalization
- Multi-Task Deep Neural Networks for Natural Language Understanding
- ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension
- Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding
- A Surprisingly Robust Trick for Winograd Schema Challenge
- Probing Neural Network Comprehension of Natural Language Arguments
Cited by in corpus (5)
- DeBERTa: Decoding-enhanced BERT with Disentangled Attention
- WinoGrande: An Adversarial Winograd Schema Challenge at Scale
- Extending Automated Deduction for Commonsense Reasoning
- CUHK at SemEval-2020 Task 4: CommonSense Explanation, Reasoning and Prediction with Multi-task Learning
- Unsupervised Pronoun Resolution via Masked Noun-Phrase Prediction