Publications (4)
Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency
Jan Batzner, Volker Stocker, Bingjun Tang +4
Synthetic personae experiments have become a prominent method in Large Language Model alignment research, yet the representativeness and ecological validity of these personae vary…
Beyond Citations: Measuring Idea-level Knowledge Diffusion from Research to Journalism and Policy-making
Yangliu Fan, Kilian Buehling, Volker Stocker
Despite the importance of social science knowledge for various stakeholders, measuring its diffusion into different domains remains a challenge. This study uses a novel text-based…
Sycophancy Claims about Language Models: The Missing Human-in-the-Loop
Jan Batzner, Volker Stocker, Stefan Schmid +1
Sycophantic response patterns in Large Language Models (LLMs) have been increasingly claimed in the literature. We review methodological challenges in measuring LLM sycophancy and…
GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancy
Jan Batzner, Volker Stocker, Stefan Schmid +1
Large language models (LLMs) are increasingly shaping citizens' information ecosystems. Products incorporating LLMs, such as chatbots and AI Companions, are now widely used for dec…