12 citations · 32 across the 7 of their papers we have counts for
7 papers
Real-Time Visual Feedback to Guide Benchmark Creation: A Human-and-Metric-in-the-Loop Workflow
Anjana Arunkumar, Swaroop Mishra, Bhavdeep Sachdeva +2
Recent research has shown that language models exploit `artifacts' in benchmarks to solve tasks, rather than truly learning them, leading to inflated model performance. In pursuit…
Pretrained Transformers Do not Always Improve Robustness
Swaroop Mishra, Bhavdeep Singh Sachdeva, Chitta Baral
Pretrained Transformers (PT) have been shown to improve Out of Distribution (OOD) robustness than traditional models such as Bag of Words (BOW), LSTMs, Convolutional Neural Network…
NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks
Swaroop Mishra, Arindam Mitra, Neeraj Varshney +4
Given the ubiquitous nature of numbers in text, reasoning with numbers to perform simple calculations is an important skill of AI systems. While many datasets and models have been…
Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness
Tejas Gokhale, Swaroop Mishra, Man Luo +2
Data modification, either via additional training datasets, data augmentation, debiasing, and dataset filtering, has been proposed as an effective solution for generalizing to out-…
DQI: A Guide to Benchmark Evaluation
Swaroop Mishra, Anjana Arunkumar, Bhavdeep Sachdeva +2
A `state of the art' model A surpasses humans in a benchmark B, but fails on similar benchmarks C, D, and E. What does B have that the other benchmarks do not? Recent research prov…
Towards Question Format Independent Numerical Reasoning: A Set of Prerequisite Tasks
Swaroop Mishra, Arindam Mitra, Neeraj Varshney +2
Numerical reasoning is often important to accurately understand the world. Recently, several format-specific datasets have been proposed, such as numerical reasoning in the setting…