1 citations · 2 across the 17 of their papers we have counts for
18 papers
Evaluating the Semantic Specificity of Representation Steering in Language Models
Zhangdie Yuan, Andreas Vlachos
Localized Representation Steering (LRS) is widely used to correct reasoning pathologies in large language models. However, standard benchmark evaluations can easily be fooled by su…
Self-Exploring Language Models for Explainable Link Forecasting on Temporal Graphs via Reinforcement Learning
Zifeng Ding, Shenyang Huang, Zeyu Cao +11
Forecasting future links is a central task in temporal graph (TG) reasoning, requiring models to leverage historical interactions to predict upcoming ones. Traditional neural appro…
Do Language Models Update their Forecasts with New Information?
Zhangdie Yuan, Zifeng Ding, Andreas Vlachos
Prior work has largely treated future event prediction as a static task, failing to consider how forecasts should evolve as new evidence emerges. To address this gap, we introduce…
TCP: a Benchmark for Temporal Constraint-Based Planning
Zifeng Ding, Sikuan Yan, Zhangdie Yuan +3
Temporal reasoning and planning are essential capabilities for large language models (LLMs), yet most existing benchmarks evaluate them in isolation and under limited forms of comp…
Social Good or Scientific Curiosity? Uncovering the Research Framing Behind NLP Artefacts
Eric Chamoun, Nedjma Ousidhoum, Michael Schlichtkrull +1
Clarifying the research framing of NLP artefacts (e.g., models, datasets, etc.) is crucial to aligning research with practical applications. Recent studies manually analyzed NLP re…
AVerImaTeC: A Dataset for Automatic Verification of Image-Text Claims with Evidence from the Web
Rui Cao, Zifeng Ding, Zhijiang Guo +2
Textual claims are often accompanied by images to enhance their credibility and spread on social media, but this also raises concerns about the spread of misinformation. Existing d…
Capturing Symmetry and Antisymmetry in Language Models through Symmetry-Aware Training Objectives
Zhangdie Yuan, Andreas Vlachos
Capturing symmetric (e.g., country borders another country) and antisymmetric (e.g., parent_of) relations is crucial for a variety of applications. This paper tackles this challeng…
FOReCAst: The Future Outcome Reasoning and Confidence Assessment Benchmark
Zhangdie Yuan, Zifeng Ding, Andreas Vlachos
Forecasting is an important task in many domains, such as technology and economics. However existing forecasting benchmarks largely lack comprehensive confidence assessment, focus…
Segment-Level Diffusion: A Framework for Controllable Long-Form Generation with Diffusion Language Models
Xiaochen Zhu, Georgi Karadzhov, Chenxi Whitehouse +1
Diffusion models have shown promise in text generation, but often struggle with generating long, coherent, and contextually accurate text. Token-level diffusion doesn't model word-…
Decompose and Leverage Preferences from Expert Models for Improving Trustworthiness of MLLMs
Rui Cao, Yuming Jiang, Michael Schlichtkrull +1
Multimodal Large Language Models (MLLMs) can enhance trustworthiness by aligning with human preferences. As human preference labeling is laborious, recent works employ evaluation m…
TabVer: Tabular Fact Verification with Natural Logic
Rami Aly, Andreas Vlachos
Fact verification on tabular evidence incentivises the use of symbolic reasoning models where a logical form is constructed (e.g. a LISP-style program), providing greater verifiabi…
Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checking
Mubashara Akhtar, Michael Schlichtkrull, Andreas Vlachos
Current automated fact-checking (AFC) approaches typically evaluate evidence either implicitly via the predicted verdicts or through exact matches with predefined closed knowledge…
The Automated Verification of Textual Claims (AVeriTeC) Shared Task
Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse +9
The Automated Verification of Textual Claims (AVeriTeC) shared task asks participants to retrieve evidence and predict veracity for real-world claims checked by fact-checkers. Evid…
Zero-Shot Fact Verification via Natural Logic and Large Language Models
Marek Strong, Rami Aly, Andreas Vlachos
The recent development of fact verification systems with natural logic has enhanced their explainability by aligning claims with evidence through set-theoretic operators, providing…
Document-level Claim Extraction and Decontextualisation for Fact-Checking
Zhenyun Deng, Michael Schlichtkrull, Andreas Vlachos
Selecting which claims to check is a time-consuming task for human fact-checkers, especially from documents consisting of multiple sentences and containing multiple claims. However…
Automated Focused Feedback Generation for Scientific Writing Assistance
Eric Chamoun, Michael Schlichktrull, Andreas Vlachos
Scientific writing is a challenging task, particularly for novice researchers who often rely on feedback from experienced peers. Recent work has primarily focused on improving surf…
PRobELM: Plausibility Ranking Evaluation for Language Models
Zhangdie Yuan, Eric Chamoun, Rami Aly +2
This paper introduces PRobELM (Plausibility Ranking Evaluation for Language Models), a benchmark designed to assess language models' ability to discern more plausible from less pla…
Zero-Shot Fact-Checking with Semantic Triples and Knowledge Graphs
Zhangdie Yuan, Andreas Vlachos
Despite progress in automated fact-checking, most systems require a significant amount of labeled training data, which is expensive. In this paper, we propose a novel zero-shot met…