9 papers
From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness
Qianli Wang, Yilong Wang, Dennis Wei +5
Chain-of-thought (CoT) can sound plausible yet be unfaithful to the model's underlying reasoning. Most prior work probes CoT faithfulness through input--output behavior or input at…
Investigating the Interplay between Contextual and Parametric Chain-of-Thought Faithfulness under Optimization
Jingyi Sun, Qianli Wang, Pepa Atanasova +2
Chain-of-Thought (CoT) faithfulness, i.e., whether CoTs genuinely reflect large language models' (LLM) underlying behavior, is typically evaluated with metrics under two disjoint p…
Show me the evidence: Evaluating the role of evidence and natural language explanations in AI-supported fact-checking
Greta Warren, Jingyi Sun, Irina Shklovski +1
Although much research has focused on AI explanations to support decisions in complex information-seeking tasks such as fact-checking, the role of evidence is surprisingly under-re…
Spark-Prover-X1: Formal Theorem Proving Through Diverse Data Training
Xinyuan Zhou, Yi Lei, Xiaoyu Zhou +7
Large Language Models (LLMs) have shown significant promise in automated theorem proving, yet progress is often constrained by the scarcity of diverse and high-quality formal langu…
Evaluation Framework for Highlight Explanations of Context Utilisation in Language Models
Jingyi Sun, Pepa Atanasova, Sagnik Ray Choudhury +2
Context utilisation, the ability of Language Models (LMs) to incorporate relevant information from the provided context when generating responses, remains largely opaque to users,…
Explaining Sources of Uncertainty in Automated Fact-Checking
Jingyi Sun, Greta Warren, Irina Shklovski +1
Understanding sources of a model's uncertainty regarding its predictions is crucial for effective human-AI collaboration. Prior work proposes using numerical uncertainty or hedges…
Graph-Guided Textual Explanation Generation Framework
Shuzhou Yuan, Jingyi Sun, Ran Zhang +4
Natural language explanations (NLEs) are commonly used to provide plausible free-text explanations of a model's reasoning about its predictions. However, recent work has questioned…
Evaluating Input Feature Explanations through a Unified Diagnostic Evaluation Framework
Jingyi Sun, Pepa Atanasova, Isabelle Augenstein
Explaining the decision-making process of machine learning models is crucial for ensuring their reliability and transparency for end users. One popular explanation form highlights…
Stand for Something or Fall for Everything: Predict Misinformation Spread with Stance-Aware Graph Neural Networks
Zihan Chen, Jingyi Sun, Rong Liu +1
Although pervasive spread of misinformation on social media platforms has become a pressing challenge, existing platform interventions have shown limited success in curbing its dis…