9 papers
Social Bias or Social Desirability? Comparing Factorial Survey Experiments for Humans and Large Language Models
Franziska Weeber, Thomas Hinz, Konstantin Mozer +3
Factorial survey experiments are a survey method from the social sciences that uses descriptions of socially complex situation (``vignettes'') to elicit judgments such as social bi…
Effects of Answer Format Variation on Gender Bias in Large Language Models
Ksenia Merzlyakova, Sebastian Padó, Franziska Weeber
Gender bias or other social biases in large language models (LLMs) are frequently evaluated with question answering or survey benchmarks where the LLM needs to give a response in a…
Understanding the Impact of Linguistic Realization Choices on LLM Stance with Causal Tracing
Langchen Huang, Sebastian Padó, Franziska Weeber
Large language models (LLMs) are known to be sensitive to prompt and input formulations. However, existing studies have focused on lexical realization and largely ignored construct…
AmchiBias: Measuring Stereotypical Bias in Goan Identity Groups with a Minimal Pair Dataset in English and Konkani
Michelle Barbosa, Sebastian Padó, Franziska Weeber
Socio-cultural stereotypical bias is an important consideration in the development and deployment of NLP systems. It is however often considered only at the national level, despite…
One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalization
Franziska Weeber, Vera Neplenbroek, Jan Batzner +1
Personalization of LLMs by sociodemographic subgroup often improves user experience, but can also introduce or amplify biases and unfair outcomes across groups. Prior work has empl…
Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMs
Tristan Williams, Franziska Weeber, Sebastian Padó +1
Large language models are increasingly used to represent human opinions, values, or beliefs, and their steerability towards these ideals is an active area of research. Existing wor…
PRSM: A Measure to Evaluate CLIP's Robustness Against Paraphrases
Udo Schlegel, Franziska Weeber, Jian Lan +1
Contrastive Language-Image Pre-training (CLIP) is a widely used multimodal model that aligns text and image representations through large-scale training. While it performs strongly…
Do Political Opinions Transfer Between Western Languages? An Analysis of Unaligned and Aligned Multilingual LLMs
Franziska Weeber, Tanise Ceron, Sebastian Padó
Public opinion surveys show cross-cultural differences in political opinions between socio-cultural contexts. However, there is no clear evidence whether these differences translat…
Assisted Text Annotation Using Active Learning to Achieve High Quality with Little Effort
Franziska Weeber, Felix Hamborg, Karsten Donnay +1
Large amounts of annotated data have become more important than ever, especially since the rise of deep learning techniques. However, manual annotations are costly. We propose a to…