12 citations · 12 across the 18 of their papers we have counts for
12 papers · 1 filter
When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness
Mahdi Dhaini, Adam Dejl, Juraj Vladika +3
Natural language explanations (NLEs) are increasingly used as inputs, for example, as few-shot rationales that influence model behavior in in-context learning (ICL). However, it re…
BavGround: A Benchmark for Regional Cultural Grounding and Dialect Competence in Bavarian
Jophin John, Michael Hoffmann, Jan Fillies +2
Cultural evaluation of large language models (LLMs) often focuses on high-resource standard languages, leaving regional culture and dialect communities underrepresented. We introdu…
Reasoning that Travels: Dissecting How Chain-of-Thought Transfers Across Models
Xinyuan Cheng, Beiduo Chen, Philipp Mondorf +1
Large reasoning models (LRMs) often generate extensive chain-of-thought (CoT) traces before producing a final answer. As explicit textual artifacts, these traces can be passed to o…
If Probable, Then Acceptable? Understanding Conditional Acceptability Judgments in Large Language Models
Jasmin Orth, Philipp Mondorf, Barbara Plank
Conditional acceptability refers to how plausible a conditional statement is perceived to be. It plays an important role in communication and reasoning, as it influences how indivi…
What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns
Michael A. Hedderich, Anyi Wang, Raoyuan Zhao +3
Prompt engineering for large language models is challenging, as even small prompt perturbations or model changes can significantly impact the generated output texts. Existing evalu…
MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMs
Raoyuan Zhao, Beiduo Chen, Barbara Plank +1
Large language models (LLMs) are used globally across many languages, but their English-centric pretraining raises concerns about cross-lingual disparities for cultural awareness,…
Probing LLMs for Multilingual Discourse Generalization Through a Unified Label Set
Florian Eichin, Yang Janet Liu, Barbara Plank +1
Discourse understanding is essential for many NLP tasks, yet most existing work remains constrained by framework-dependent discourse representations. This work investigates whether…
The Validation Gap: A Mechanistic Analysis of How Language Models Compute Arithmetic but Fail to Validate It
Leonardo Bertolazzi, Philipp Mondorf, Barbara Plank +1
The ability of large language models (LLMs) to validate their output and identify potential errors is crucial for ensuring robustness and reliability. However, current research ind…
Understanding When Tree of Thoughts Succeeds: Larger Models Excel in Generation, Not Discrimination
Qiqi Chen, Xinpeng Wang, Philipp Mondorf +2
Tree of Thoughts (ToT) is a reasoning strategy for Large Language Models (LLMs) that employs a generator to suggest reasoning steps and a discriminator to decide which steps to imp…
Liar, Liar, Logical Mire: A Benchmark for Suppositional Reasoning in Large Language Models
Philipp Mondorf, Barbara Plank
Knights and knaves problems represent a classic genre of logical puzzles where characters either tell the truth or lie. The objective is to logically deduce each character's identi…
Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
Philipp Mondorf, Barbara Plank
Large language models (LLMs) have recently shown impressive performance on tasks involving reasoning, leading to a lively debate on whether these models possess reasoning capabilit…
Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoning
Philipp Mondorf, Barbara Plank
Deductive reasoning plays a pivotal role in the formulation of sound and cohesive arguments. It allows individuals to draw conclusions that logically follow, given the truth value…