25 citations · 34 across the 12 of their papers we have counts for
13 papers
POOL: Propagated Uncertainty Over Lookalikes
Rounak Sharma, Ananya B. Sai, Soumyabrata Pal
Black-box large language models need confidence scores that can separate likely-correct from likely-incorrect outputs, enabling systems to prioritize human review, route uncertain…
A Good Talk Does not Look Like a Summary, It Teaches You! Measuring Takeaways from Paper-to-Video Talks
Ishani Mondal, Aparna Garimella, Ananya Sai +2
Automatically generated videos from scientific papers are increasingly used for education and research dissemination. However, existing evaluation metrics mainly measure visual qua…
How Good is Zero-Shot MT Evaluation for Low Resource Indian Languages?
Anushka Singh, Ananya B. Sai, Raj Dabre +3
While machine translation evaluation has been studied primarily for high-resource languages, there has been a recent interest in evaluation for low-resource languages due to the in…
Language Models can Subtly Deceive Without Lying: A Case Study on Strategic Phrasing in Legislation
Atharvan Dogra, Krishna Pillutla, Ameet Deshpande +5
We explore the ability of large language models (LLMs) to engage in subtle deception through strategically phrasing and intentionally manipulating information. This harmful behavio…
Closing the Gap in the Trade-off between Fair Representations and Accuracy
Biswajit Rout, Ananya B. Sai, Arun Rajkumar
The rapid developments of various machine learning models and their deployments in several applications has led to discussions around the importance of looking beyond the accuracie…
BiPhone: Modeling Inter Language Phonetic Influences in Text
Abhirut Gupta, Ananya B. Sai, Richard Sproat +5
A large number of people are forced to use the Web in a language they have low literacy in due to technology asymmetries. Written text in the second language (L2) from such users o…
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Kaustubh D. Dhole, Varun Gangal, Sebastian Gehrmann +122
Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained…
Perturbation CheckLists for Evaluating NLG Evaluation Metrics
Ananya B. Sai, Tanay Dixit, Dev Yashpal Sheth +2
Natural Language Generation (NLG) evaluation is a multifaceted task requiring assessment of multiple desirable criteria, e.g., fluency, coherency, coverage, relevance, adequacy, ov…
Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretraining
Ananya B. Sai, Akash Kumar Mohankumar, Siddhartha Arora +1
There is an increasing focus on model-based dialog evaluation metrics such as ADEM, RUBER, and the more recent BERT-based metrics. These models aim to assign a high score to all re…
A Survey of Evaluation Metrics Used for NLG Systems
Ananya B. Sai, Akash Kumar Mohankumar, Mitesh M. Khapra
The success of Deep Learning has created a surge in interest in a wide a range of Natural Language Generation (NLG) tasks. Deep Learning has not only pushed the state of the art in…
Frustratingly Poor Performance of Reading Comprehension Models on Non-adversarial Examples
Soham Parikh, Ananya B. Sai, Preksha Nema +1
When humans learn to perform a difficult task (say, reading comprehension (RC) over longer passages), it is typically the case that their performance improves significantly on an e…
ElimiNet: A Model for Eliminating Options for Reading Comprehension with Multiple Choice Questions
Soham Parikh, Ananya B. Sai, Preksha Nema +1
The task of Reading Comprehension with Multiple Choice Questions, requires a human (or machine) to read a given passage, question pair and select one of the n given options. The cu…
Re-evaluating ADEM: A Deeper Look at Scoring Dialogue Responses
Ananya B. Sai, Mithun Das Gupta, Mitesh M. Khapra +1
Automatically evaluating the quality of dialogue responses for unstructured domains is a challenging problem. ADEM(Lowe et al. 2017) formulated the automatic evaluation of dialogue…