2 citations · 5 across the 13 of their papers we have counts for
11 papers · 1 filter
Evaluating the Semantic Specificity of Representation Steering in Language Models
Zhangdie Yuan, Andreas Vlachos
Localized Representation Steering (LRS) is widely used to correct reasoning pathologies in large language models. However, standard benchmark evaluations can easily be fooled by su…
Toward Reliable Clinical Coding with Language Models: Verification and Lightweight Adaptation
Zhangdie Yuan, Han-Chin Shing, Mitch Strong +1
Accurate clinical coding is essential for healthcare documentation, billing, and decision-making. While prior work shows that off-the-shelf LLMs struggle with this task, evaluation…
Do Language Models Update their Forecasts with New Information?
Zhangdie Yuan, Zifeng Ding, Andreas Vlachos
Prior work has largely treated future event prediction as a static task, failing to consider how forecasts should evolve as new evidence emerges. To address this gap, we introduce…
Capturing Symmetry and Antisymmetry in Language Models through Symmetry-Aware Training Objectives
Zhangdie Yuan, Andreas Vlachos
Capturing symmetric (e.g., country borders another country) and antisymmetric (e.g., parent_of) relations is crucial for a variety of applications. This paper tackles this challeng…
PRobELM: Plausibility Ranking Evaluation for Language Models
Zhangdie Yuan, Eric Chamoun, Rami Aly +2
This paper introduces PRobELM (Plausibility Ranking Evaluation for Language Models), a benchmark designed to assess language models' ability to discern more plausible from less pla…
DIALIGHT: Lightweight Multilingual Development and Evaluation of Task-Oriented Dialogue Systems with Large Language Models
Songbo Hu, Xiaobin Wang, Zhangdie Yuan +2
We present DIALIGHT, a toolkit for developing and evaluating multilingual Task-Oriented Dialogue (ToD) systems which facilitates systematic evaluations and comparisons between ToD…
Language Models as Hierarchy Encoders
Yuan He, Zhangdie Yuan, Jiaoyan Chen +1
Interpreting hierarchical structures latent in language is a key limitation of current language models (LMs). While previous research has implicitly leveraged these hierarchies to…
Zero-Shot Fact-Checking with Semantic Triples and Knowledge Graphs
Zhangdie Yuan, Andreas Vlachos
Despite progress in automated fact-checking, most systems require a significant amount of labeled training data, which is expensive. In this paper, we propose a novel zero-shot met…
A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems
Songbo Hu, Han Zhou, Moy Yuan +5
Achieving robust language technologies that can perform well across the world's many languages is a central goal of multilingual NLP. In this work, we take stock of and empirically…
Varifocal Question Generation for Fact-checking
Nedjma Ousidhoum, Zhangdie Yuan, Andreas Vlachos
Fact-checking requires retrieving evidence related to a claim under investigation. The task can be formulated as question generation based on a claim, followed by question answerin…
Can Pretrained Language Models (Yet) Reason Deductively?
Zhangdie Yuan, Songbo Hu, Ivan Vulić +2
Acquiring factual knowledge with Pretrained Language Models (PLMs) has attracted increasing attention, showing promising performance in many knowledge-intensive tasks. Their good p…