11 papers · 1 filter
From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness
Qianli Wang, Yilong Wang, Dennis Wei +5
Chain-of-thought (CoT) can sound plausible yet be unfaithful to the model's underlying reasoning. Most prior work probes CoT faithfulness through input--output behavior or input at…
Limitations of Automated Simulatability: LLM Simulators Can Bypass Explanations
Antonin Poché, Fanny Jourdan, Nils Feldhus +6
Simulatability is an evaluation protocol for explanations that quantifies their usefulness by how well they help a user predict a task model's outputs. Since human evaluation is co…
Investigating the Interplay between Contextual and Parametric Chain-of-Thought Faithfulness under Optimization
Jingyi Sun, Qianli Wang, Pepa Atanasova +2
Chain-of-Thought (CoT) faithfulness, i.e., whether CoTs genuinely reflect large language models' (LLM) underlying behavior, is typically evaluated with metrics under two disjoint p…
Judge Circuits Explain Format-Induced Inconsistency in LLM-as-a-Judge
Nils Feldhus, Tanja Baeumel, Elena Golimblevskaia +10
LLM-as-a-judge has become the dominant paradigm for grading model outputs at scale, yet the same model assigns systematically different scores when its output format changes (e.g.,…
Macro: Enhancing Multilingual Counterfactual Explanations through Alignment-as-Preference Optimization
Yilong Wang, Qianli Wang, Bohao Chu +3
Self-generated counterfactual explanations (SCEs) are minimally modified inputs (minimality) generated by large language models (LLMs) that flip their own predictions (validity), o…
eTracer: Polarity-Aware Evidence Grounding
Bohao Chu, Qianli Wang, Hendrik Damm +8
Evidence grounding makes generated content verifiable by linking statements to their sources. However, effective verifiability requires two properties that existing methods fail to…
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models
Hengyuan Zhang, Zhihao Zhang, Mingyang Wang +26
Mechanistic Interpretability (MI) has emerged as a vital approach to demystify the opaque decision-making of Large Language Models (LLMs). However, existing reviews primarily treat…
What Are We Measuring in NLG? A Meta-Analysis of Evaluation Trends 2020-2025
Jing Yang, Nils Feldhus, Salar Mohtaj +10
As Natural Language Generation (NLG) dominates modern NLP, scalable evaluation remains a critical bottleneck. Consequently, LLM-as-a-judge (LaaJ) adoption has accelerated rapidly,…
iFlip: Iterative Feedback-driven Counterfactual Example Refinement
Yilong Wang, Qianli Wang, Nils Feldhus
Counterfactual examples are minimal edits to an input that alter a model's prediction. They are widely employed in explainable AI to probe model behavior and in natural language pr…
Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanations
Qianli Wang, Nils Feldhus, Pepa Atanasova +4
Quantization is widely used to accelerate inference and streamline the deployment of large language models (LLMs), yet its effects on self-explanations (SEs) remain unexplored. SEs…
Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation
Qianli Wang, Van Bach Nguyen, Yihong Liu +6
Counterfactuals refer to minimally edited inputs that cause a model's prediction to change, serving as a promising approach to explaining the model's behavior. Large language model…