6 papers
Studying People to Study AI: Expert Perspectives on the Epistemic Fit and Barriers of Human Research in AI Safety & Ethics
Jessica Y. Bo, Paula Akemi Aoyagui, Shalaleh Rismani +3
Safety risks of AI are becoming increasingly evident in human interactions with AI technologies. The prominent approaches to evaluating these risks favor technical methods, such as…
Large Language Lovers: Lived Experiences of Negotiating Agency and Platform Control in AI Companionship
Patrick Yung Kang Lee, Jessica Y. Bo, Zixin Zhao +6
Individuals are turning to increasingly anthropomorphic, general-purpose chatbots for AI companionship, rather than roleplay-specific platforms. However, not much is known about ho…
DICE: A Framework for Dimensional and Contextual Evaluation of Language Models
Aryan Shrivastava, Paula Akemi Aoyagui
Language models (LMs) are increasingly being integrated into a wide range of applications, yet the modern evaluation paradigm does not sufficiently reflect how they are actually be…
A Matter of Perspective(s): Contrasting Human and LLM Argumentation in Subjective Decision-Making on Subtle Sexism
Paula Akemi Aoyagui, Kelsey Stemmler, Sharon Ferguson +2
In subjective decision-making, where decisions are based on contextual interpretation, Large Language Models (LLMs) can be integrated to present users with additional rationales to…
Exploring Subjectivity for more Human-Centric Assessment of Social Biases in Large Language Models
Paula Akemi Aoyagui, Sharon Ferguson, Anastasia Kuzminykh
An essential aspect of evaluating Large Language Models (LLMs) is identifying potential biases. This is especially relevant considering the substantial evidence that LLMs can repli…
Just Like Me: The Role of Opinions and Personal Experiences in The Perception of Explanations in Subjective Decision-Making
Sharon Ferguson, Paula Akemi Aoyagui, Young-Ho Kim +1
As large language models (LLMs) advance to produce human-like arguments in some contexts, the number of settings applicable for human-AI collaboration broadens. Specifically, we fo…