47 citations · 91 across the 20 of their papers we have counts for
6 papers · 1 filter
Hardness of Samples Need to be Quantified for a Reliable Evaluation System: Exploring Potential Opportunities with a New Task
Swaroop Mishra, Anjana Arunkumar, Chris Bryan +1
Evaluation of models on benchmarks is unreliable without knowing the degree of sample hardness; this subsequently overestimates the capability of AI systems and limits their adopti…
A Survey of Parameters Associated with the Quality of Benchmarks in NLP
Swaroop Mishra, Anjana Arunkumar, Chris Bryan +1
Several benchmarks have been built with heavy investment in resources to track our progress in NLP. Thousands of papers published in response to those benchmarks have competed to t…
Investigating the Failure Modes of the AUC metric and Exploring Alternatives for Evaluating Systems in Safety Critical Applications
Swaroop Mishra, Anjana Arunkumar, Chitta Baral
With the increasing importance of safety requirements associated with the use of black box models, evaluation of selective answering capability of models has been critical. Area un…
PMU Tracker: A Visualization Platform for Epicentric Event Propagation Analysis in the Power Grid
Anjana Arunkumar, Andrea Pinceti, Lalitha Sankar +1
The electrical power grid is a critical infrastructure, with disruptions in transmission having severe repercussions on daily activities, across multiple sectors. To identify, prev…
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi +37
How well can NLP models generalize to a variety of unseen tasks when provided with task instructions? To address this question, we first introduce Super-NaturalInstructions, a benc…
A Proposal to Study "Is High Quality Data All We Need?"
Swaroop Mishra, Anjana Arunkumar
Even though deep neural models have achieved superhuman performance on many popular benchmarks, they have failed to generalize to OOD or adversarial datasets. Conventional approach…