3 papers
cs.AI2026
Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior
Rafal Kocielnik, Pengrui Han, Peiyang Song +5
Anticipating LLM behavioral tendencies from low-cost psychometric probes is critical for safe deployment, but only if self-reports (SR) reliably predict behavior. Recent work docum…
cs.CL2026
Estimating Causal Effects of Text Interventions Leveraging LLMs
Siyi Guo, Myrl G. Marmarelis, Fred Morstatter +1
Quantifying the effects of textual interventions in social systems, such as reducing anger in social media posts to see its impact on engagement, is challenging. Real-world interve…
cs.CL2024
Capturing Perspectives of Crowdsourced Annotators in Subjective Learning Tasks
Negar Mokhberian, Myrl G. Marmarelis, Frederic R. Hopp +3
Supervised classification heavily depends on datasets annotated by humans. However, in subjective tasks such as toxicity classification, these annotations often exhibit low agreeme…