1 citations · 2 across the 10 of their papers we have counts for
10 papers
TriQua: Reconciling Granularity and Context in Factuality Evaluation
Jin Liu, Steffen Thoma, Achim Rettinger
The "decompose-then-verify" paradigm for LLM factuality evaluation faces a fundamental trade-off: atomic facts, i.e., one sentence conveying one unit of information, often omit ess…
The Unsampled Truth: Quantifying Prompt Artifacts in LM Psychometrics
Nils Schwager, Christoph Hau, Simon Münker +1
When prompting language models for psychometric assessment, researchers assume that the responses reflect the injected persona and the meaning of the survey item. We test this prem…
LLM Agents Predict Social Media Reactions but Do Not Outperform Text Classifiers: Benchmarking Simulation Accuracy Using 120K+ Personas of 1511 Humans
Ljubisa Bojic, Alexander Felfernig, Bojana Dinic +4
Social media platforms mediate how billions form opinions and engage with public discourse. As autonomous AI agents increasingly participate in these spaces, understanding their be…
Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction
Nils Schwager, Simon Münker, Alistair Plum +1
The transition of Large Language Models (LLMs) from exploratory tools to active "silicon subjects" in social science lacks extensive validation of operational validity. This study…
Next Reply Prediction X Dataset: Linguistic Discrepancies in Naively Generated Content
Simon Münker, Nils Schwager, Kai Kugler +2
The increasing use of Large Language Models (LLMs) as proxies for human participants in social science research presents a promising, yet methodologically risky, paradigm shift. Wh…
Identity-Aware Large Language Models require Cultural Reasoning
Alistair Plum, Anne-Marie Lutgen, Christoph Purschke +1
Large language models have become the latest trend in natural language processing, heavily featuring in the digital tools we use every day. However, their replies often reflect a n…
Don't Trust Generative Agents to Mimic Communication on Social Networks Unless You Benchmarked their Empirical Realism
Simon Münker, Nils Schwager, Achim Rettinger
The ability of Large Language Models (LLMs) to mimic human behavior triggered a plethora of computational social science research, assuming that empirical studies of humans can be…
Zero-shot prompt-based classification: topic labeling in times of foundation models in German Tweets
Simon Münker, Kai Kugler, Achim Rettinger
Filtering and annotating textual data are routine tasks in many areas, like social media or news analytics. Automating these tasks allows to scale the analyses wrt. speed and bread…
POV Learning: Individual Alignment of Multimodal Models using Human Perception
Simon Werner, Katharina Christ, Laura Bernardy +2
Aligning machine learning systems with human expectations is mostly attempted by training with manually vetted human behavioral samples, typically explicit feedback. This is done o…
InvBERT: Reconstructing Text from Contextualized Word Embeddings by inverting the BERT pipeline
Kai Kugler, Simon Münker, Johannes Höhmann +1
Digital Humanities and Computational Literary Studies apply text mining methods to investigate literature. Such automated approaches enable quantitative studies on large corpora wh…