The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants

Abstract

Reasoning is a crucial part of natural language argumentation. To comprehend an argument, one must analyze its warrant, which explains why its claim follows from its premises. As arguments are highly contextualized, warrants are usually presupposed and left implicit. Thus, the comprehension does not only require language understanding and logic skills, but also depends on common sense. In this paper we develop a methodology for reconstructing warrants systematically. We operationalize it in a scalable crowdsourcing process, resulting in a freely licensed dataset with warrants for 2k authentic arguments from news comments. On this basis, we present a new challenging task, the argument reasoning comprehension task. Given an argument with a claim and a premise, the goal is to choose the correct implicit warrant from two options. Both warrants are plausible and lexically close, but lead to contradicting claims. A solution to this task will define a substantial step towards automatic warrant reconstruction. However, experiments with several neural attention and language models reveal that current approaches do not suffice.

Accepted as NAACL 2018 Long Paper; see details on the front page

References in corpus (1)

• Skip-Thought Vectors

Cited by in corpus (20)

• The Debate Over Understanding in AI's Large Language Models
• ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning
• A Retrospective Analysis of the Fake News Challenge Stance Detection Task
• Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches
• The evolution of argumentation mining: From models to social media and emerging tools
• Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
• Probing Neural Network Comprehension of Natural Language Arguments
• Topic-Guided Sampling For Data-Efficient Multi-Domain Stance Detection
• Accurate, yet inconsistent? Consistency Analysis on Language Understanding Models
• Implicit Knowledge in Argumentative Texts: An Annotated Corpus
• Causal Question Answering with Reinforcement Learning
• LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
• When Choosing Plausible Alternatives, Clever Hans can be Clever
• CO-NNECT: A Framework for Revealing Commonsense Knowledge Paths as Explicitations of Implicit Knowledge in Texts
• Non-Axiomatic Term Logic: A Computational Theory of Cognitive Symbolic Reasoning
• Implicit Premise Generation with Discourse-aware Commonsense Knowledge Models
• Riposte! A Large Corpus of Counter-Arguments
• Tribrid: Stance Classification with Neural Inconsistency Detection
• Human-in-the-Loop Disinformation Detection: Stance, Sentiment, or Something Else?
• MultiOpEd: A Corpus of Multi-Perspective News Editorials