10 citations · 17 across the 17 of their papers we have counts for
18 papers
Prompt Revision as a Source of Cultural Bias in Text-to-Image Systems
Aleksandra Urman, Elsa Lichtenegger, Salima Jaoua +7
Commercial text-to-image systems silently revise user prompts before generating images, a step users typically cannot disable or even see. Yet, existing audits of cultural bias exa…
API Benchmark Scores Do Not Reliably Transfer to Chatbot Interfaces
Jennifer Wang, Joachim Baumann, Daniel E. Ho +1
Benchmark scores are a central currency in model releases: they inform purchasing decisions, shape public trust, and influence policy. Yet, a key assumption underlying benchmark sc…
WildSEEK: Evaluating Language Models for Information-Seeking
Tanise Ceron, Joachim Baumann, Elisa Bassignana +3
Language models are increasingly mediating information access to end users, urging a systematic evaluation of their responses for a fair and reliable information ecosystem. Existin…
Humanly: A Configurable and Traceable Environment for Human-AI Collaborative Writing
Shenzhe Zhu, Haoqian Zhang, Xu Yang +7
Teachers, conference chairs, and public readers all judge writing from limited evidence, seeing only a finished document and not the process that produced it. Final text alone cann…
Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability
Alicia Parrish, Rajat Shinde, Sanket Badhe +57
Current AI safety evaluation and benchmarking frameworks predominantly rely on Western-centric culture-agnostic defaults that mask critical regional laws, socio-linguistic nuances,…
SecureForge: Finding and Preventing Vulnerabilities in LLM-Generated Code via Prompt Optimization
Houjun Liu, Lisa Einstein, John Yang +5
LLM coding agents now generate code at an unprecedented scale, yet LLM-generated code introduces cybersecurity vulnerabilities into codebases without human involvement. Even when f…
Stop Automating Peer Review Without Rigorous Evaluation
Joachim Baumann, Jiaxin Pei, Sanmi Koyejo +1
Large language models offer a tempting solution to address the peer review crisis. This position paper argues that today's AI systems should not be used to produce paper reviews. W…
SWE-chat: Coding Agent Interactions From Real Users in the Wild
Joachim Baumann, Vishakh Padmakumar, Xiang Li +3
AI coding agents are being adopted at scale, yet we lack empirical evidence on how people actually use them and how much of their output is useful in practice. We present SWE-chat,…
GoogleTrendArchive: A Year-Long Archive of Real-Time Web Search Trends Worldwide
Aleksandra Urman, Anikó Hannák, Joachim Baumann
GoogleTrendArchive is a comprehensive archive of Google Trending Now data spanning over one year (from November 28, 2024 to January 3, 2026) across 125 countries and 1,358 location…
Auditing Google's AI Overviews and Featured Snippets: A Case Study on Baby Care and Pregnancy
Desheng Hu, Joachim Baumann, Aleksandra Urman +4
Google Search increasingly surfaces AI-generated content through features like AI Overviews (AIO) and Featured Snippets (FS), which users frequently rely on despite having no contr…
Reduced AI Acceptance After the Generative AI Boom: Evidence From a Two-Wave Survey Study
Joachim Baumann, Aleksandra Urman, Ulrich Leicht-Deobald +3
The rapid adoption of generative artificial intelligence (GenAI) technologies has led many organizations to integrate AI into their products and services, often without considering…
Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation
Joachim Baumann, Paul Röttger, Aleksandra Urman +4
Large language models are rapidly transforming social science research by enabling the automation of labor-intensive tasks like data annotation and text analysis. However, LLM outp…
On the Societal Impact of Machine Learning
Joachim Baumann
This PhD thesis investigates the societal impact of machine learning (ML). ML increasingly informs consequential decisions and recommendations, significantly affecting many aspects…
SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors
Tiancheng Hu, Joachim Baumann, Lorenzo Lupo +3
Large language model (LLM) simulations of human behavior have the potential to revolutionize the social and behavioral sciences, if and only if they faithfully reflect real human b…
AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons
Shaona Ghosh, Heather Frase, Adina Williams +99
The rapid advancement and deployment of AI systems have created an urgent need for standard safety-evaluation frameworks. This paper introduces AILuminate v1.0, the first comprehen…
Algorithmic Collective Action in Recommender Systems: Promoting Songs by Reordering Playlists
Joachim Baumann, Celestine Mendler-Dünner
We investigate algorithmic collective action in transformer-based recommender systems. Our use case is a music streaming platform where a collective of fans aims to promote the vis…
Preventing Eviction-Caused Homelessness through ML-Informed Distribution of Rental Assistance
Catalina Vajiac, Arun Frey, Joachim Baumann +5
Rental assistance programs provide individuals with financial assistance to prevent housing instabilities caused by evictions and avert homelessness. Since these programs operate u…
On Prediction-Modelers and Decision-Makers: Why Fairness Requires More Than a Fair Prediction Model
Teresa Scantamburlo, Joachim Baumann, Christoph Heitz
An implicit ambiguity in the field of prediction-based decision-making regards the relation between the concepts of prediction and decision. Much of the literature in the field ten…