3 citations · 3 across the 15 of their papers we have counts for
16 papers
Bayesian Fine-tuning Yields Language Models that are as Bayesian as their Beliefs Allow
Polina Tsvilodub, Andreas Waldis, Linlu Qiu +2
Language models (LMs) are increasingly used for tasks that require reasoning about hidden variables from a few observations, for which Bayesian inference is the normatively correct…
CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition
Lance Ying, Jinzhou Wu, Yingshan Susan Wang +53
Understanding and modeling human intelligence are parallel goals shared by artificial intelligence (AI) and cognitive science. As AI systems grow increasingly capable, in what ways…
Confirming Our Biases? Evaluating the Capabilities, Risks, and Societal Impact of Large Language Models
Mudar Adas, Polina Tsvilodub, Michael Franke +1
It is well established that large language models (LLMs) are sensitive to prompt framing, reflecting patterns in their training data or prior prompts. In this study, we investigate…
Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Models
Polina Tsvilodub, Max Höth, Michael Franke +2
While modern large reasoning models (LRMs) excel at providing correct answers in many tasks, we provide additional evidence that they often struggle with a critical capability: kno…
Computational models of pragmatic reasoning with flexible generation of meaning and expression alternatives
Polina Tsvilodub, Fausto Carcassi, Michael Franke
Pragmatic language use requires reasoning about alternatives: the alternative expressions a speaker might have chosen, or the alternative interpretations a listener might entertain…
Post-training makes large language models less human-like
Marcel Binz, Elif Akata, Abdullah Almaatouq +76
Large language models (LLMs) are increasingly used as surrogates for human participants, but it remains unclear which models best capture human behavior and why. To address this, w…
On Emergent Social World Models -- Evidence for Functional Integration of Theory of Mind and Pragmatic Reasoning in Language Models
Polina Tsvilodub, Jan-Felix Klumpp, Amir Mohammadpour +2
This paper investigates whether LMs recruit shared computational mechanisms for general Theory of Mind (ToM) and language-specific pragmatic reasoning in order to contribute to the…
Act or Clarify? Modeling Sensitivity to Uncertainty and Cost in Communication
Polina Tsvilodub, Karl Mulligan, Todd Snider +2
When deciding how to act under uncertainty, agents may choose to act to reduce uncertainty or they may act despite that uncertainty. In communicative settings, an important way of…
Integrating Neural and Symbolic Components in a Model of Pragmatic Question-Answering
Polina Tsvilodub, Robert D. Hawkins, Michael Franke
Computational models of pragmatic language use have traditionally relied on hand-specified sets of utterances and meanings, limiting their applicability to real-world language use.…
Non-literal Understanding of Number Words by Language Models
Polina Tsvilodub, Kanishk Gandhi, Haoran Zhao +3
Humans naturally interpret numbers non-literally, effortlessly combining context, world knowledge, and speaker intent. We investigate whether large language models (LLMs) interpret…
Cognitive Modeling with Scaffolded LLMs: A Case Study of Referential Expression Generation
Polina Tsvilodub, Michael Franke, Fausto Carcassi
To what extent can LLMs be used as part of a cognitive model of language generation? In this paper, we approach this question by exploring a neuro-symbolic implementation of an alg…
Bayesian Statistical Modeling with Predictors from LLMs
Michael Franke, Polina Tsvilodub, Fausto Carcassi
State of the art large language models (LLMs) have shown impressive performance on a variety of benchmark tasks and are increasingly used as components in larger applications, wher…
Experimental Pragmatics with Machines: Testing LLM Predictions for the Inferences of Plain and Embedded Disjunctions
Polina Tsvilodub, Paul Marty, Sonia Ramotowska +2
Human communication is based on a variety of inferences that we draw from sentences, often going beyond what is literally said. While there is wide agreement on the basic distincti…
Predictions from language models for multiple-choice tasks are not robust under variation of scoring methods
Polina Tsvilodub, Hening Wang, Sharon Grosch +1
This paper systematically compares different methods of deriving item-level predictions of language models for multiple-choice tasks. It compares scoring methods for answer options…
Evaluating Pragmatic Abilities of Image Captioners on A3DS
Polina Tsvilodub, Michael Franke
Evaluating grounded neural language model performance with respect to pragmatic qualities like the trade off between truthfulness, contrastivity and overinformativity of generated…
Overinformative Question Answering by Humans and Machines
Polina Tsvilodub, Michael Franke, Robert D. Hawkins +1
When faced with a polar question, speakers often provide overinformative answers going beyond a simple "yes" or "no". But what principles guide the selection of additional informat…