11 papers
Social Bias or Social Desirability? Comparing Factorial Survey Experiments for Humans and Large Language Models
Franziska Weeber, Thomas Hinz, Konstantin Mozer +3
Factorial survey experiments are a survey method from the social sciences that uses descriptions of socially complex situation (``vignettes'') to elicit judgments such as social bi…
Effects of Answer Format Variation on Gender Bias in Large Language Models
Ksenia Merzlyakova, Sebastian Padó, Franziska Weeber
Gender bias or other social biases in large language models (LLMs) are frequently evaluated with question answering or survey benchmarks where the LLM needs to give a response in a…
Off-the-Shelf Language-Vision Encoders Predict Human Gaze Preferences in Visual World Experiments
Rahul Murali Shankar, Titus von der Malsburg, Sebastian Padó
A central question for computational psycholinguistics is to what extent neural language models can serve as cognitively adequate models of human language processing. However, work…
Understanding the Impact of Linguistic Realization Choices on LLM Stance with Causal Tracing
Langchen Huang, Sebastian Padó, Franziska Weeber
Large language models (LLMs) are known to be sensitive to prompt and input formulations. However, existing studies have focused on lexical realization and largely ignored construct…
Diverging Transformer Predictions for Human Sentence Processing: A Comprehensive Analysis of Agreement Attraction Effects
Titus von der Malsburg, Sebastian Padó
Transformers underlie almost all state-of-the-art language models in computational linguistics, yet their cognitive adequacy as models of human sentence processing remains disputed…
Finding Sense in Nonsense with Generated Contexts: Perspectives from Humans and Language Models
Katrina Olsen, Sebastian Padó
Nonsensical and anomalous sentences have been instrumental in the development of computational models of semantic interpretation. A core challenge is to distinguish between what is…
One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalization
Franziska Weeber, Vera Neplenbroek, Jan Batzner +1
Personalization of LLMs by sociodemographic subgroup often improves user experience, but can also introduce or amplify biases and unfair outcomes across groups. Prior work has empl…
Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMs
Tristan Williams, Franziska Weeber, Sebastian Padó +1
Large language models are increasingly used to represent human opinions, values, or beliefs, and their steerability towards these ideals is an active area of research. Existing wor…
Democratizing News Recommenders: Modeling Multiple Perspectives for News Candidate Generation with VQ-VAE
Hardy, Sebastian Padó, Amelie Wührl +1
News Recommender Systems (NRS) shape what users read, whose perspectives they encounter, and influence public discourse. Yet their design is value-laden: intentionally or not, NRS…
Do Political Opinions Transfer Between Western Languages? An Analysis of Unaligned and Aligned Multilingual LLMs
Franziska Weeber, Tanise Ceron, Sebastian Padó
Public opinion surveys show cross-cultural differences in political opinions between socio-cultural contexts. However, there is no clear evidence whether these differences translat…
Efficient Language Modeling for Low-Resource Settings with Hybrid RNN-Transformer Architectures
Gabriel Lindenmaier, Sean Papay, Sebastian Padó
Transformer-based language models have recently been at the forefront of active research in text generation. However, these models' advances come at the price of prohibitive traini…