298 citations · 657 across the 9 of their papers we have counts for
9 papers
NodeSynth: Socially Aligned Synthetic Data for AI Evaluation
Qazi Mamunur Rashid, Xuan Yang, Zhengzhe Yang +5
Recent advancements in generative AI facilitate large-scale synthetic data generation for model evaluation. However, without targeted approaches, these datasets often lack the soci…
Cultural Perspectives and Expectations for Generative AI: A Global Survey Approach
Erin van Liemt, Renee Shelby, Andrew Smart +5
There is a lack of empirical evidence about global attitudes around whether and how GenAI should represent cultures. This paper assesses understandings and beliefs about culture as…
Amplify Initiative: Building A Localized Data Platform for Globalized AI
Qazi Mamunur Rashid, Erin van Liemt, Tiffany Shih +19
Current AI models often fail to account for local context and language, given the predominance of English and Western internet content in their training data. This hinders the glob…
Socially Responsible Data for Large Multilingual Language Models
Andrew Smart, Ben Hutchinson, Lameck Mbangula Amugongo +12
Large Language Models (LLMs) have rapidly increased in size and apparent capabilities in the last three years, but their training data is largely English text. There is growing int…
A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models
Stephen R. Pfohl, Heather Cole-Lewis, Rory Sayres +27
Large language models (LLMs) hold promise to serve complex health information needs but also have the potential to introduce harm and exacerbate health disparities. Reliably evalua…
How Knowledge Workers Think Generative AI Will (Not) Transform Their Industries
Allison Woodruff, Renee Shelby, Patrick Gage Kelley +3
Generative AI is expected to have transformative effects in multiple knowledge industries. To better understand how knowledge workers expect generative AI may affect their industri…
The Equitable AI Research Roundtable (EARR): Towards Community-Based Decision Making in Responsible AI Development
Jamila Smith-Loud, Andrew Smart, Darlene Neal +14
This paper reports on our initial evaluation of The Equitable AI Research Roundtable -- a coalition of experts in law, education, community engagement, social justice, and technolo…
Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing
Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White +6
Rising concern for the societal implications of artificial intelligence systems has inspired a wave of academic and journalistic literature in which deployed systems are audited fo…
Towards a Critical Race Methodology in Algorithmic Fairness
Alex Hanna, Emily Denton, Andrew Smart +1
We examine the way race and racial categories are adopted in algorithmic fairness frameworks. Current methodologies fail to adequately account for the socially constructed nature o…