565 citations · 1.2k across the 49 of their papers we have counts for
5 papers · 1 filter
DQI: A Guide to Benchmark Evaluation
Swaroop Mishra, Anjana Arunkumar, Bhavdeep Sachdeva +2
A `state of the art' model A surpasses humans in a benchmark B, but fails on similar benchmarks C, D, and E. What does B have that the other benchmarks do not? Recent research prov…
Towards Improving Selective Prediction Ability of NLP Systems
Neeraj Varshney, Swaroop Mishra, Chitta Baral
It's better to say "I can't answer" than to answer incorrectly. This selective prediction ability is crucial for NLP systems to be reliably deployed in real-world applications. Pri…
Our Evaluation Metric Needs an Update to Encourage Generalization
Swaroop Mishra, Anjana Arunkumar, Chris Bryan +1
Models that surpass human performance on several popular benchmarks display significant degradation in performance on exposure to Out of Distribution (OOD) data. Recent research ha…
Towards Question Format Independent Numerical Reasoning: A Set of Prerequisite Tasks
Swaroop Mishra, Arindam Mitra, Neeraj Varshney +2
Numerical reasoning is often important to accurately understand the world. Recently, several format-specific datasets have been proposed, such as numerical reasoning in the setting…
DQI: Measuring Data Quality in NLP
Swaroop Mishra, Anjana Arunkumar, Bhavdeep Sachdeva +2
Neural language models have achieved human level performance across several NLP datasets. However, recent studies have shown that these models are not truly learning the desired ta…