14 citations · 32 across the 9 of their papers we have counts for
9 papers
The Privacy-Hallucination Tradeoff in Differentially Private Language Models
Krithika Ramesh, Krishna Pillutla, Danish Pruthi +1
Both privacy and factual accuracy are paramount in high-stakes domains like healthcare. Concerningly, we uncover and investigate a privacy-hallucination tradeoff in differentially…
How do we measure privacy in text? A survey of text anonymization metrics
Yaxuan Ren, Krithika Ramesh, Yaxing Yao +1
In this work, we aim to clarify and reconcile metrics for evaluating privacy protection in text through a systematic survey. Although text anonymization is essential for enabling N…
SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domains
Krithika Ramesh, Daniel Smolyak, Zihao Zhao +4
We present SynthTextEval, a toolkit for conducting comprehensive evaluations of synthetic text. The fluency of large language model (LLM) outputs has made synthetic text potentiall…
Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains
Krithika Ramesh, Nupoor Gandhi, Pulkit Madaan +3
The difficulty of anonymizing text data hinders the development and deployment of NLP in high-stakes domains that involve private data, such as healthcare and social services. Poor…
MEGA: Multilingual Evaluation of Generative AI
Kabir Ahuja, Harshita Diddee, Rishav Hada +9
Generative AI models have shown impressive performance on many Natural Language Processing tasks such as language understanding, reasoning, and language generation. An important qu…
Fairness in Language Models Beyond English: Gaps and Challenges
Krithika Ramesh, Sunayana Sitaram, Monojit Choudhury
With language models becoming increasingly ubiquitous, it has become essential to address their inequitable treatment of diverse demographic groups and factors. Most research on ev…
'Beach' to 'Bitch': Inadvertent Unsafe Transcription of Kids' Content on YouTube
Krithika Ramesh, Ashiqur R. KhudaBukhsh, Sumeet Kumar
Over the last few years, YouTube Kids has emerged as one of the highly competitive alternatives to television for children's entertainment. Consequently, YouTube Kids' content shou…
Towards Quantifying the Carbon Emissions of Differentially Private Machine Learning
Rakshit Naidu, Harshita Diddee, Ajinkya Mulay +3
In recent years, machine learning techniques utilizing large-scale datasets have achieved remarkable performance. Differential privacy, by means of adding noise, provides strong pr…
Evaluating Gender Bias in Hindi-English Machine Translation
Gauri Gupta, Krithika Ramesh, Sanjay Singh
With language models being deployed increasingly in the real world, it is essential to address the issue of the fairness of their outputs. The word embedding representations of the…