11 citations · 30 across the 23 of their papers we have counts for
23 papers
The Privacy-Hallucination Tradeoff in Differentially Private Language Models
Krithika Ramesh, Krishna Pillutla, Danish Pruthi +1
Both privacy and factual accuracy are paramount in high-stakes domains like healthcare. Concerningly, we uncover and investigate a privacy-hallucination tradeoff in differentially…
Expectations and Practices around AI Disclosure in CS Research
Arati Mohapatra, Danish Pruthi
As generative AI tools find increasing use in research workflows, ongoing debates on their impact, appropriateness and responsible use have led policymakers to enact policies to di…
Evaluating Reasoning Models for Queries with Presuppositions
Rose Sathyanathan, Kinshuk Vasisht, Danish Pruthi
Millions of users turn to AI models for their information needs. It is conceivable that a large number of user queries contain assumptions that may be factually inaccurate. Prior w…
Policies Permitting LLM Use for Polishing Peer Reviews Are Currently Not Enforceable
Rounak Saha, Gurusha Juneja, Dayita Chaudhuri +3
A number of scientific conferences and journals have recently enacted policies that prohibit LLM usage by peer reviewers, except for polishing, paraphrasing, and grammar correction…
Where Do Images Come From? Analyzing Captions to Geographically Profile Datasets
Abhipsa Basu, Yugam Bahl, Kirti Bhagat +3
Recent studies show that text-to-image models often fail to generate geographically representative images, raising concerns about the representativeness of their training data and…
Beyond World Models: Rethinking Understanding in AI Models
Tarun Gupta, Danish Pruthi
World models have garnered substantial interest in the AI community. These are internal representations that simulate aspects of the external world, track entities and states, capt…
TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Stories
Kirti Bhagat, Shaily Bhatt, Athul Velagapudi +3
Millions of users across the globe turn to AI chatbots for their creative needs, inviting widespread interest in understanding how they represent diverse cultures. However, evaluat…
FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypes
Janki Atul Nawale, Mohammed Safi Ur Rahman Khan, Janani D +3
Existing studies on fairness are largely Western-focused, making them inadequate for culturally diverse countries such as India. To address this gap, we introduce INDIC-BIAS, a com…
Silencing Empowerment, Allowing Bigotry: Auditing the Moderation of Hate Speech on Twitch
Prarabdh Shukla, Wei Yin Chong, Yash Patel +3
To meet the demands of content moderation, online platforms have resorted to automated systems. Newer forms of real-time engagement(, users commenting on live stream…
All That Glitters is Not Novel: Plagiarism in AI Generated Research
Tarun Gupta, Danish Pruthi
Automating scientific research is considered the final frontier of science. Recently, several papers claim autonomous research agents can generate novel research ideas. Amidst the…
Knowledge Graph Guided Evaluation of Abstention Techniques
Kinshuk Vasisht, Navreet Kaur, Danish Pruthi
To deploy language models safely, it is crucial that they abstain from responding to inappropriate requests. Several prior studies test the safety promises of models based on their…
Revisiting the Robustness of Watermarking to Paraphrasing Attacks
Saksham Rastogi, Danish Pruthi
Amidst rising concerns about the internet being proliferated with content generated from language models (LMs), watermarking is seen as a principled way to certify whether text was…
Richer Output for Richer Countries: Uncovering Geographical Disparities in Generated Stories and Travel Recommendations
Kirti Bhagat, Kinshuk Vasisht, Danish Pruthi
While a large body of work inspects language models for biases concerning gender, race, occupation and religion, biases of geographical nature are relatively less explored. Some re…
What Can Natural Language Processing Do for Peer Review?
Ilia Kuznetsov, Osama Mohammed Afzal, Koen Dercksen +21
The number of scientific articles produced every year is growing rapidly. Providing quality control over them is crucial for scientists and, ultimately, for the public good. In mod…
Evaluating Large Language Models for Health-related Queries with Presuppositions
Navreet Kaur, Monojit Choudhury, Danish Pruthi
As corporations rush to integrate large language models (LLMs) to their search offerings, it is critical that they provide factually accurate information that is robust to any pres…
Downstream Trade-offs of a Family of Text Watermarks
Anirudh Ajith, Sameer Singh, Danish Pruthi
Watermarking involves implanting an imperceptible signal into generated text that can later be detected via statistical tests. A prominent family of watermarking strategies for LLM…
Geographical Erasure in Language Generation
Pola Schwöbel, Jacek Golebiowski, Michele Donini +2
Large language models (LLMs) encode vast amounts of world knowledge. However, since these models are trained on large swaths of internet data, they are at risk of inordinately capt…
Inspecting the Geographical Representativeness of Images from Text-to-Image Models
Abhipsa Basu, R. Venkatesh Babu, Danish Pruthi
Recent progress in generative models has resulted in models that produce both realistic as well as relevant images for most textual inputs. These models are being used to generate…
Assisting Human Decisions in Document Matching
Joon Sik Kim, Valerie Chen, Danish Pruthi +2
Many practical applications, ranging from paper-reviewer assignment in peer review to job-applicant matching for hiring, require human decision makers to identify relevant matches…
Measures of Information Reflect Memorization Patterns
Rachit Bansal, Danish Pruthi, Yonatan Belinkov
Neural networks are known to exploit spurious artifacts (or shortcuts) that co-occur with a target label, exhibiting heuristic memorization. On the other hand, networks have been s…
Learning to Scaffold: Optimizing Model Explanations for Teaching
Patrick Fernandes, Marcos Treviso, Danish Pruthi +2
Modern machine learning models are opaque, and as a result there is a burgeoning academic subfield on methods that explain these models' behavior. However, what is the precise goal…
Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations
Siddhant Arora, Danish Pruthi, Norman Sadeh +3
In attempts to "explain" predictions of machine learning models, researchers have proposed hundreds of techniques for attributing predictions to features that are deemed important.…
Do Context-Aware Translation Models Pay the Right Attention?
Kayo Yin, Patrick Fernandes, Danish Pruthi +3
Context-aware machine translation models are designed to leverage contextual information, but often fail to do so. As a result, they inaccurately disambiguate pronouns and polysemo…