4 papers
No PUN Intended: Plausible Unknown Names for Person-Centred LLM Evaluation
Dimitri Staufer, David Hartmann, Ibrahim Baroud
Person names are widely used as prompt variables in LLM evaluations of factuality, privacy leakage, bias and abstention, but when a name's evidential status is uncontrolled, measur…
Bye Bye Perspective API: Lessons for Measurement Infrastructure in NLP, CSS and LLM Evaluation
David Hartmann, Manuel Tonneau, Angelie Kraft +7
The closure of Perspective API at the end of 2026 discards what has functioned as the de facto standard for automated toxicity measurement in NLP, CSS, and LLM evaluation research.…
Human-Centred LLM Privacy Audits: Findings and Frictions
Dimitri Staufer, Kirsten Morehouse, David Hartmann +1
Large language models (LLMs) learn statistical associations from massive training corpora and user interactions, and deployed systems can surface or infer information about individ…
Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations
David Hartmann, Amin Oueslati, Dimitri Staufer +3
Commercial content moderation APIs are marketed as scalable solutions to combat online hate speech. However, the reliance on these APIs risks both silencing legitimate speech, call…