2 citations · 5 across the 13 of their papers we have counts for
14 papers
Evaluating the Semantic Specificity of Representation Steering in Language Models
Zhangdie Yuan, Andreas Vlachos
Localized Representation Steering (LRS) is widely used to correct reasoning pathologies in large language models. However, standard benchmark evaluations can easily be fooled by su…
Self-Exploring Language Models for Explainable Link Forecasting on Temporal Graphs via Reinforcement Learning
Zifeng Ding, Shenyang Huang, Zeyu Cao +11
Forecasting future links is a central task in temporal graph (TG) reasoning, requiring models to leverage historical interactions to predict upcoming ones. Traditional neural appro…
Toward Reliable Clinical Coding with Language Models: Verification and Lightweight Adaptation
Zhangdie Yuan, Han-Chin Shing, Mitch Strong +1
Accurate clinical coding is essential for healthcare documentation, billing, and decision-making. While prior work shows that off-the-shelf LLMs struggle with this task, evaluation…
Do Language Models Update their Forecasts with New Information?
Zhangdie Yuan, Zifeng Ding, Andreas Vlachos
Prior work has largely treated future event prediction as a static task, failing to consider how forecasts should evolve as new evidence emerges. To address this gap, we introduce…
TCP: a Benchmark for Temporal Constraint-Based Planning
Zifeng Ding, Sikuan Yan, Zhangdie Yuan +3
Temporal reasoning and planning are essential capabilities for large language models (LLMs), yet most existing benchmarks evaluate them in isolation and under limited forms of comp…
Capturing Symmetry and Antisymmetry in Language Models through Symmetry-Aware Training Objectives
Zhangdie Yuan, Andreas Vlachos
Capturing symmetric (e.g., country borders another country) and antisymmetric (e.g., parent_of) relations is crucial for a variety of applications. This paper tackles this challeng…
FOReCAst: The Future Outcome Reasoning and Confidence Assessment Benchmark
Zhangdie Yuan, Zifeng Ding, Andreas Vlachos
Forecasting is an important task in many domains, such as technology and economics. However existing forecasting benchmarks largely lack comprehensive confidence assessment, focus…
PRobELM: Plausibility Ranking Evaluation for Language Models
Zhangdie Yuan, Eric Chamoun, Rami Aly +2
This paper introduces PRobELM (Plausibility Ranking Evaluation for Language Models), a benchmark designed to assess language models' ability to discern more plausible from less pla…
DIALIGHT: Lightweight Multilingual Development and Evaluation of Task-Oriented Dialogue Systems with Large Language Models
Songbo Hu, Xiaobin Wang, Zhangdie Yuan +2
We present DIALIGHT, a toolkit for developing and evaluating multilingual Task-Oriented Dialogue (ToD) systems which facilitates systematic evaluations and comparisons between ToD…
Language Models as Hierarchy Encoders
Yuan He, Zhangdie Yuan, Jiaoyan Chen +1
Interpreting hierarchical structures latent in language is a key limitation of current language models (LMs). While previous research has implicitly leveraged these hierarchies to…
Zero-Shot Fact-Checking with Semantic Triples and Knowledge Graphs
Zhangdie Yuan, Andreas Vlachos
Despite progress in automated fact-checking, most systems require a significant amount of labeled training data, which is expensive. In this paper, we propose a novel zero-shot met…
A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems
Songbo Hu, Han Zhou, Moy Yuan +5
Achieving robust language technologies that can perform well across the world's many languages is a central goal of multilingual NLP. In this work, we take stock of and empirically…
Varifocal Question Generation for Fact-checking
Nedjma Ousidhoum, Zhangdie Yuan, Andreas Vlachos
Fact-checking requires retrieving evidence related to a claim under investigation. The task can be formulated as question generation based on a claim, followed by question answerin…
Can Pretrained Language Models (Yet) Reason Deductively?
Zhangdie Yuan, Songbo Hu, Ivan Vulić +2
Acquiring factual knowledge with Pretrained Language Models (PLMs) has attracted increasing attention, showing promising performance in many knowledge-intensive tasks. Their good p…