71 citations · 184 across the 23 of their papers we have counts for
7 papers · 1 filter
Easy, Reproducible and Quality-Controlled Data Collection with Crowdaq
Qiang Ning, Hao Wu, Pradeep Dasigi +5
High-quality and large-scale data are key to success for AI systems. However, large-scale data annotation efforts are often confronted with a set of common challenges: (1) designin…
Temporal Reasoning on Implicit Events from Distant Supervision
Ben Zhou, Kyle Richardson, Qiang Ning +3
We propose TRACIE, a novel temporal reasoning dataset that evaluates the degree to which systems understand implicit events -- events that are not mentioned explicitly in natural l…
Learnability with Indirect Supervision Signals
Kaifu Wang, Qiang Ning, Dan Roth
Learning from indirect supervision signals is important in real-world AI applications when, often, gold labels are missing or too costly. In this paper, we develop a unified theore…
Foreseeing the Benefits of Incidental Supervision
Hangfeng He, Mingyuan Zhang, Qiang Ning +1
Real-world applications often require improved models by leveraging a range of cheap incidental supervision signals. These could include partial labels, noisy labels, knowledge-bas…
Temporal Common Sense Acquisition with Minimal Supervision
Ben Zhou, Qiang Ning, Daniel Khashabi +1
Temporal common sense (e.g., duration and frequency of events) is crucial for understanding natural language. However, its acquisition is challenging, partly because such informati…
TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions
Qiang Ning, Hao Wu, Rujun Han +3
A critical part of reading is being able to understand the temporal relationships between events described in a passage of text, even when those relationships are not explicitly st…
Evaluating Models' Local Decision Boundaries via Contrast Sets
Matt Gardner, Yoav Artzi, Victoria Basmova +23
Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluation…