4 papers
Human-Centred LLM Privacy Audits: Findings and Frictions
Dimitri Staufer, Kirsten Morehouse, David Hartmann +1
Large language models (LLMs) learn statistical associations from massive training corpora and user interactions, and deployed systems can surface or infer information about individ…
What Do LLMs Associate with Your Name? A Human-Centered Black-Box Audit of Personal Data
Dimitri Staufer, Kirsten Morehouse
Large language models (LLMs), and conversational agents based on them, are exposed to personal data (PD) during pre-training and during user interactions. Prior work shows that PD…
AI Benchmark Democratization and Carpentry
Gregor von Laszewski, Wesley Brewer, Jeyan Thiyagalingam +28
Benchmarks are a cornerstone of modern machine learning, enabling reproducibility, comparison, and scientific progress. However, AI benchmarks are increasingly complex, requiring d…
Rethinking LLM Bias Probing Using Lessons from the Social Sciences
Kirsten N. Morehouse, Siddharth Swaroop, Weiwei Pan
The proliferation of LLM bias probes introduces three significant challenges: (1) we lack principled criteria for choosing appropriate probes, (2) we lack a system for reconciling…