7 papers
Long-term Measurements: Towards a Longitudinal Understanding of Human-AI Interactions
Nicole Mitchell, Dhruv Agarwal, Maty Bohacek +2
Language models have taken on the role of a very new type of technology, by virtue of their "human-ness" and rapid integration into users' daily lives. This combination of features…
PLURAL: A Global Dataset for Value Alignment
Dhruv Agarwal, Anya Shukla, Tanya Goyal +1
Large language models (LLMs) are used worldwide, yet disproportionately reflect Western values, limiting their ability to represent diverse value systems. We introduce PLURAL, a la…
AI-Assisted Systematization for Evaluating GenAI Systems
Dhruv Agarwal, Emily Sheng, Chad Atalla +6
Evaluating generative AI (GenAI) systems is challenging because many targets of evaluation are broad, contested concepts, such as "reasoning," "fairness," or "creativity." When the…
Fluent but Foreign: Even Regional LLMs Lack Cultural Alignment
Dhruv Agarwal, Anya Shukla, Sunayana Sitaram +1
Large language models (LLMs) are used worldwide, yet exhibit Western cultural tendencies. Many countries are now building ``regional'' or ``sovereign'' LLMs, but it remains unclear…
Steering AI-Driven Personalization of Scientific Text for General Audiences
Taewook Kim, Dhruv Agarwal, Jordan Ackerman +1
Digital media platforms (e.g., science blogs) offer opportunities to communicate scientific content to general audiences at scale. However, these audiences vary in their scientific…
AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances
Dhruv Agarwal, Mor Naaman, Aditya Vashistha
Large language models (LLMs) are being increasingly integrated into everyday products and services, such as coding tools and writing assistants. As these embedded AI applications a…