61 citations · 78 across the 4 of their papers we have counts for
6 papers
Recognizing Atari and Assessing Sente: Controlled Evaluation Tasks for Language-Based Go Agents
Taiyi Chen, Iskar Deng, Amandalynne Paullada
Domain terminology provides concise descriptions of game states and strategic relationships, but whether game-playing language models can accurately recognize these concepts and re…
Before the Labels: How Dataset Construction Shapes Suicidality Detection in Clinical Text
Priyanshi Garg, Ishita Rao, Jieqiong Ding +1
Clinical NLP increasingly relies on electronic health record (EHR) data to detect suicidal behaviors, treating clinical documentation as more reliable ground truth than social medi…
Behind the Mask: Demographic bias in name detection for PII masking
Courtney Mansfield, Amandalynne Paullada, Kristen Howell
Many datasets contain personally identifiable information, or PII, which poses privacy risks to individuals. PII masking is commonly used to redact personal information such as nam…
AI and the Everything in the Whole Wide World Benchmark
Inioluwa Deborah Raji, Emily M. Bender, Amandalynne Paullada +2
There is a tendency across different subfields in AI to valorize a small collection of influential benchmarks. These benchmarks operate as stand-ins for a range of anointed common…
A multilabel approach to morphosyntactic probing
Naomi Tachikawa Shapiro, Amandalynne Paullada, Shane Steinert-Threlkeld
We introduce a multilabel probing task to assess the morphosyntactic representations of word embeddings from multilingual language models. We demonstrate this task with multilingua…
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Amandalynne Paullada, Inioluwa Deborah Raji, Emily M. Bender +2
Datasets have played a foundational role in the advancement of machine learning research. They form the basis for the models we design and deploy, as well as our primary medium for…