12 papers
Prompt Stability Scoring for Text Annotation with Large Language Models
Christopher Barrie, Elli Palaiologou, Petter Törnberg
Researchers are increasingly using language models (LMs) for text annotation. These approaches rely only on a prompt telling the model to return a given output according to a set o…
Political Bias Audits of LLMs Capture Sycophancy to the Inferred Auditor
Petter Törnberg, Michelle Schimmel
Large language models (LLMs) are commonly evaluated for political bias based on their responses to fixed questionnaires, which typically place frontier models on the political left…
Polarization by Default: Auditing Recommendation Bias in LLM-Based Content Curation
Nicolò Pagan, Christopher Barrie, Chris Andrew Bail +1
Large Language Models (LLMs) are increasingly deployed to curate and rank human-created content, yet the nature and structure of their biases in these tasks remains poorly understo…
Large Language Models Reproduce Racial Stereotypes When Used for Text Annotation
Petter Törnberg, Petter Törnberg
Large language models (LLMs) are increasingly used for automated text annotation in tasks ranging from academic research to content moderation and hiring. Across 19 LLMs and two ex…
Computational Turing Test Reveals Systematic Differences Between Human and AI Language
Nicolò Pagan, Petter Törnberg, Christopher A. Bail +2
Large language models (LLMs) are increasingly used in the social sciences to simulate human behavior, based on the assumption that they can generate realistic, human-like text. Yet…
Shifts in U.S. Social Media Use, 2020-2024: Decline, Fragmentation, and Enduring Polarization
Petter Törnberg
Using nationally representative data from the 2020 and 2024 American National Election Studies (ANES), this paper traces how the U.S. social media landscape has shifted across plat…