20 papers
From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness
Qianli Wang, Yilong Wang, Dennis Wei +5
Chain-of-thought (CoT) can sound plausible yet be unfaithful to the model's underlying reasoning. Most prior work probes CoT faithfulness through input--output behavior or input at…
Limitations of Automated Simulatability: LLM Simulators Can Bypass Explanations
Antonin Poché, Fanny Jourdan, Nils Feldhus +6
Simulatability is an evaluation protocol for explanations that quantifies their usefulness by how well they help a user predict a task model's outputs. Since human evaluation is co…
Investigating the Interplay between Contextual and Parametric Chain-of-Thought Faithfulness under Optimization
Jingyi Sun, Qianli Wang, Pepa Atanasova +2
Chain-of-Thought (CoT) faithfulness, i.e., whether CoTs genuinely reflect large language models' (LLM) underlying behavior, is typically evaluated with metrics under two disjoint p…
Judge Circuits Explain Format-Induced Inconsistency in LLM-as-a-Judge
Nils Feldhus, Tanja Baeumel, Elena Golimblevskaia +10
LLM-as-a-judge has become the dominant paradigm for grading model outputs at scale, yet the same model assigns systematically different scores when its output format changes (e.g.,…
Macro: Enhancing Multilingual Counterfactual Explanations through Alignment-as-Preference Optimization
Yilong Wang, Qianli Wang, Bohao Chu +3
Self-generated counterfactual explanations (SCEs) are minimally modified inputs (minimality) generated by large language models (LLMs) that flip their own predictions (validity), o…
eTracer: Polarity-Aware Evidence Grounding
Bohao Chu, Qianli Wang, Hendrik Damm +8
Evidence grounding makes generated content verifiable by linking statements to their sources. However, effective verifiability requires two properties that existing methods fail to…
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models
Hengyuan Zhang, Zhihao Zhang, Mingyang Wang +26
Mechanistic Interpretability (MI) has emerged as a vital approach to demystify the opaque decision-making of Large Language Models (LLMs). However, existing reviews primarily treat…
What Are We Measuring in NLG? A Meta-Analysis of Evaluation Trends 2020-2025
Jing Yang, Nils Feldhus, Salar Mohtaj +10
As Natural Language Generation (NLG) dominates modern NLP, scalable evaluation remains a critical bottleneck. Consequently, LLM-as-a-judge (LaaJ) adoption has accelerated rapidly,…
iFlip: Iterative Feedback-driven Counterfactual Example Refinement
Yilong Wang, Qianli Wang, Nils Feldhus
Counterfactual examples are minimal edits to an input that alter a model's prediction. They are widely employed in explainable AI to probe model behavior and in natural language pr…
Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanations
Qianli Wang, Nils Feldhus, Pepa Atanasova +4
Quantization is widely used to accelerate inference and streamline the deployment of large language models (LLMs), yet its effects on self-explanations (SEs) remain unexplored. SEs…
Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation
Qianli Wang, Van Bach Nguyen, Yihong Liu +6
Counterfactuals refer to minimally edited inputs that cause a model's prediction to change, serving as a promising approach to explaining the model's behavior. Large language model…
Multilingual Datasets for Custom Input Extraction and Explanation Requests Parsing in Conversational XAI Systems
Qianli Wang, Tatiana Anikina, Nils Feldhus +6
Conversational explainable artificial intelligence (ConvXAI) systems based on large language models (LLMs) have garnered considerable attention for their ability to enhance user co…
Truth or Twist? Optimal Model Selection for Reliable Label Flipping Evaluation in LLM-based Counterfactuals
Qianli Wang, Van Bach Nguyen, Nils Feldhus +4
Counterfactual examples are widely employed to enhance the performance and robustness of large language models (LLMs) through counterfactual data augmentation (CDA). However, the s…
Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recall
Qianli Wang, Mingyang Wang, Nils Feldhus +5
Quantization methods are widely used to accelerate inference and streamline the deployment of large language models (LLMs). Although quantization's effects on various LLM capabilit…
FitCF: A Framework for Automatic Feature Importance-guided Counterfactual Example Generation
Qianli Wang, Nils Feldhus, Simon Ostermann +3
Counterfactual examples are widely used in natural language processing (NLP) as valuable data to improve models, and in explainable artificial intelligence (XAI) to understand mode…
Anchored Alignment for Self-Explanations Enhancement
Luis Felipe Villa-Arenas, Ata Nizamoglu, Qianli Wang +2
In this work, we introduce a methodology for alignment designed to enhance the ability of large language models (LLMs) to articulate their reasoning (self-explanation) even in the…
Cross-Refine: Improving Natural Language Explanation Generation by Learning in Tandem
Qianli Wang, Tatiana Anikina, Nils Feldhus +3
Natural language explanations (NLEs) are vital for elucidating the reasoning behind large language model (LLM) decisions. Many techniques have been developed to generate NLEs using…
CoXQL: A Dataset for Parsing Explanation Requests in Conversational XAI Systems
Qianli Wang, Tatiana Anikina, Nils Feldhus +2
Conversational explainable artificial intelligence (ConvXAI) systems based on large language models (LLMs) have garnered significant interest from the research community in natural…
LLMCheckup: Conversational Examination of Large Language Models via Interpretability Tools and Self-Explanations
Qianli Wang, Tatiana Anikina, Nils Feldhus +3
Interpretability tools that offer explanations in the form of a dialogue have demonstrated their efficacy in enhancing users' understanding (Slack et al., 2023; Shen et al., 2023),…
InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations
Nils Feldhus, Qianli Wang, Tatiana Anikina +3
While recently developed NLP explainability methods let us open the black box in various ways (Madsen et al., 2022), a missing ingredient in this endeavor is an interactive tool of…