37 citations · 106 across the 23 of their papers we have counts for
24 papers
Accounting for Stochasticity in Studies of Large Language Model Refusal
Emma Lurie, Stephanie T. Wang, Sorelle A. Friedler +1
We present preliminary empirical evidence that single-observation queries are insufficient for evaluations of LLM refusal behaviors. Using a longitudinal auditing system, we issued…
AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence
Stephanie T. Wang, Jeffrey Gleason, Yakov Bart +2
The integration of generative AI into web search delivers synthesized answers to user queries, changing how people navigate and assess information, while raising concerns about the…
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
Ro Encarnación, Tina Behzad, Emma Lurie +1
Large language model (LLM) benchmark evaluations are routinely used to support claims about model safety, reliability, and deployment readiness. Yet most evaluations rely on a sing…
The Algorithmic Flattening of Sound: Computational Evidence and Justice Implications of AI Music Homogenization
Zoe Slendebroek, Danaé Metaxa
This paper audits whether large-scale generative music systems exhibit measurable musical homogenization relative to human-produced music, and develops a justice-centered account o…
The Beginning of ChatGPT Ads
Emma Lurie, Ro Encarnación, Sorelle A. Friedler +1
This paper presents the first empirical study of advertising content being rolled out in the user-facing online interfaces of large language models (LLMs). We systematically examin…
MonitrLLM: A Community-Centered Evaluation Infrastructure for Large Language Models
Victor Ojewale, Ro Encarnación, Suresh Venkatasubramanian +1
Benchmark suites assess model capability on controlled tasks; large-scale conversation corpora capture naturalistic use without user feedback; and in-interface feedback mechanisms…
Triangulating Across U.S. Federal AI Transparency Regimes
Emma Lurie, Emma Fauser, Qing He +2
Federal AI systems can deny benefits or flag individuals for deportation, but the public disclosures meant to make those systems visible are fragmented and unevenly detailed. This…
Do Language Models Pass the Bechdel Test? Auditing Gender Biases in LLM-Generated Screenplays
Megha N. Govindu, Stephanie T. Wang, Sorelle A. Friedler +1
As large language models (LLMs) are increasingly used in media production from journalistm to filmmaking, what impact do they have on the stories being told? Prior work has shown L…
Cost-of-Ethics Crisis: Beliefs, Decisions, and Justifications in the Job Searches of Computer Science Students in Canada and the United States
Mohamed Abdalla, Sahar Abdalla, Alicia Cappello +4
Workplace norms in computer science have received growing attention due to a series of recent ethical scandals. One response has been a push to improve the ethics education provide…
Understanding teens' self-beliefs when learning to construct and deconstruct AI/ML systems: Developing a survey instrument
Luis Morales-Navarro, Deborah Fields, Michael T. Giang +3
Despite growing calls to foster AI literacy, there are few available survey instruments designed for children and youth that study computational empowerment alongside construction…
Building to Understand: Examining Teens' Technical and Socio-Ethical Pieces of Understandings in the Construction of Small Generative Language Models
Luis Morales-Navarro, Daniel J. Noh, Lucianne Servat +3
The rising adoption of generative AI/ML technologies increases the need to support teens in developing AI/ML literacies. Child-computer interaction research argues that constructio…
"You Can Actually Do Something'': Shifts in High School Computer Science Teachers' Conceptions of AI/ML Systems and Algorithmic Justice
Daniel J. Noh, Deborah A. Fields, Yasmin B. Kafai +1
The recent proliferation of artificial intelligence and machine learning (AI/ML) systems highlights the need for all people to develop effective competencies to interact with and e…
Rapid Testing, Duck Lips, and Tilted Cameras: Youth Everyday Algorithm Auditing Practices with Generative AI Filters
Lauren Vogelstein, Vedya Konda, Deborah Fields +3
Today's youth have extensive experience interacting with artificial intelligence and machine learning applications on popular social media platforms, putting youth in a unique posi…
Longitudinal Monitoring of LLM Content Moderation of Social Issues
Yunlang Dai, Emma Lurie, Danaé Metaxa +1
Large language models' (LLMs') outputs are shaped by opaque and frequently-changing company content moderation policies and practices. LLM moderation often takes the form of refusa…
Learning AI Auditing: A Case Study of Teenagers Auditing a Generative AI Model
Luis Morales-Navarro, Michelle Gan, Evelyn Yu +3
This study investigates how high school-aged youth engage in algorithm auditing to identify and understand biases in artificial intelligence and machine learning (AI/ML) tools they…
Youth as Advisors in Participatory Design: Situating Teens' Expertise in Everyday Algorithm Auditing with Teachers and Researchers
Daniel J. Noh, Deborah A. Fields, Luis Morales-Navarro +3
Research on children and youth's participation in different roles in the design of technologies is one of the core contributions in child-computer interaction studies. Building on…
Navigating Automated Hiring: Perceptions, Strategy Use, and Outcomes Among Young Job Seekers
Lena Armstrong, Danaé Metaxa
As the use of automated employment decision tools (AEDTs) has rapidly increased in hiring contexts, especially for computing jobs, there is still limited work on applicants' percep…
Learning About Algorithm Auditing in Five Steps: Scaffolding How High School Youth Can Systematically and Critically Evaluate Machine Learning Applications
Luis Morales-Navarro, Yasmin B. Kafai, Lauren Vogelstein +2
While there is widespread interest in supporting young people to critically evaluate machine learning-powered systems, there is little research on how we can support them in inquir…
Generative AI and Perceptual Harms: Who's Suspected of using LLMs?
Kowe Kadoma, Danaé Metaxa, Mor Naaman
Large language models (LLMs) are increasingly integrated into a variety of writing tasks. While these tools can help people by generating ideas or producing higher quality work, li…
Identity-related Speech Suppression in Generative AI Content Moderation
Grace Proebsting, Oghenefejiro Isaacs Anigboro, Charlie M. Crawford +2
Automated content moderation has long been used to help identify and filter undesired user-generated content online. But such systems have a history of incorrectly flagging content…
Lower Quantity, Higher Quality: Auditing News Content and User Perceptions on Twitter/X Algorithmic versus Chronological Timelines
Stephanie Wang, Shengchun Huang, Alvin Zhou +1
Social media personalization algorithms increasingly influence the flow of civic information through society, resulting in concerns about "filter bubbles", "echo chambers", and oth…
The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiring
Lena Armstrong, Abbey Liu, Stephen MacNeil +1
Large language models (LLMs) are increasingly being introduced in workplace settings, with the goals of improving efficiency and fairness. However, concerns have arisen regarding t…
Youth as Peer Auditors: Engaging Teenagers with Algorithm Auditing of Machine Learning Applications
Luis Morales-Navarro, Yasmin B. Kafai, Vedya Konda +1
As artificial intelligence/machine learning (AI/ML) applications become more pervasive in youth lives, supporting them to interact, design, and evaluate applications is crucial. Th…
The Role of Inclusion, Control, and Ownership in Workplace AI-Mediated Communication
Kowe Kadoma, Marianne Aubin Le Quere, Jenny Fu +3
Given large language models' (LLMs) increasing integration into workplace software, it is important to examine how biases in the models may impact workers. For example, stylistic b…