Publications (105)
Multi-Trigger Poisoning Amplifies Backdoor Vulnerabilities in LLMs
Sanhanat Sivapiromrat, Caiqi Zhang, Marco Basaldella +1
Recent studies have shown that Large Language Models (LLMs) are vulnerable to data poisoning attacks, where malicious training examples embed hidden behaviours triggered by specifi…
Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders
Ivan VuliÄ, Goran GlavaÅ¡, Fangyu Liu +3
Pretrained multilingual language models (LMs) can be successfully transformed into multilingual sentence encoders (SEs; e.g., LaBSE, xMPNet) via additional fine-tuning or model dis…
A Survey on Prompt Tuning
Zongqian Li, Yixuan Su, Nigel Collier
This survey reviews prompt tuning, a parameter-efficient approach for adapting language models by prepending trainable continuous vectors while keeping the model frozen. We classif…
Unseen Word Representation by Aligning Heterogeneous Lexical Semantic Spaces
Victor Prokhorov, Mohammad Taher Pilehvar, Dimitri Kartsaklis +2
Word embedding techniques heavily rely on the abundance of training data for individual words. Given the Zipfian distribution of words in natural language texts, a large number of…
Multi-agent AI systems outperform human teams in creativity
Tiancheng Hu, Yixuan Jiang, Haotian Li +5
Although artificial intelligence (AI) now matches or exceeds human performance across numerous cognitive tasks, creativity remains a highly contested frontier. As AI systems based…
Confident Rankings with Fewer Items: Adaptive LLM Evaluation with Continuous Scores
Esma Balkır, Alice Pernthaller, Marco Basaldella +2
Computerized Adaptive Testing (CAT) has proven effective for efficient LLM evaluation on multiple-choice benchmarks, but modern LLM evaluation increasingly relies on generation tas…
Confidence Estimation for LLMs in Multi-turn Interactions
Caiqi Zhang, Ruihan Yang, Xiaochen Zhu +5
While confidence estimation is a promising direction for mitigating hallucinations in Large Language Models (LLMs), current research overwhelmingly focuses on single-turn settings.…
PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning
Zongqian Li, Yixuan Su, Nigel Collier
Parameter-efficient fine-tuning (PEFT) methods have shown promise in adapting large language models, yet existing approaches exhibit counter-intuitive phenomena: integrating router…
An Empirical Study of Sections in Classifying Disease Outbreak Reports
Son Doan, Mike Conway, Nigel Collier
Identifying articles that relate to infectious diseases is a necessary step for any automatic bio-surveillance system that monitors news articles from the Internet. Unlike scientif…
Flexi-LoRA with Input-Adaptive Ranks: Efficient Finetuning for Speech and Reasoning Tasks
Zongqian Li, Yixuan Su, Han Zhou +2
Parameter-efficient fine-tuning methods like Low-Rank Adaptation (LoRA) have become essential for deploying large language models, yet their static parameter allocation remains sub…
iNews: A Multimodal Dataset for Modeling Personalized Affective Responses to News
Tiancheng Hu, Nigel Collier
Understanding how individuals perceive and react to information is fundamental for advancing social and behavioral sciences and developing human-centered AI systems. Current approa…
Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking
Fangyu Liu, Ivan VuliÄ, Anna Korhonen +1
Injecting external domain-specific knowledge (e.g., UMLS) into pretrained language models (LMs) advances their capability to handle specialised in-domain tasks such as biomedical e…
OMG U got flu? Analysis of shared health messages for bio-surveillance
Nigel Collier, Nguyen Truong Son, Ngoc Mai Nguyen
Background: Micro-blogging services such as Twitter offer the potential to crowdsource epidemics in real-time. However, Twitter posts ('tweets') are often ambiguous and reactive to…
Do ever larger octopi still amplify reporting biases? Evidence from judgments of typical colour
Fangyu Liu, Julian Martin Eisenschlos, Jeremy R. Cole +1
Language models (LMs) trained on raw texts have no direct access to the physical world. Gordon and Van Durme (2013) point out that LMs can thus suffer from reporting bias: texts ra…
Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems
Zongqian Li, Tengchao Lv, Shaohan Huang +8
Training next-generation code generation models requires high-quality datasets, yet existing datasets face difficulty imbalance, format inconsistency, and data quality problems. We…
Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pre-trained Language Models
Zaiqiao Meng, Fangyu Liu, Ehsan Shareghi +3
Knowledge probing is crucial for understanding the knowledge transfer mechanism behind the pre-trained language models (PLMs). Despite the growing progress of probing knowledge for…
Will-They-Won't-They: A Very Large Dataset for Stance Detection on Twitter
Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar +3
We present a new challenging stance detection dataset, called Will-They-Won't-They (WT-WT), which contains 51,284 tweets in English, making it by far the largest available dataset…
Stylistic Dialogue Generation via Information-Guided Reinforcement Learning Strategy
Yixuan Su, Deng Cai, Yan Wang +4
Stylistic response generation is crucial for building an engaging dialogue system for industrial use. While it has attracted much research interest, existing methods often generate…
On the Importance of the Kullback-Leibler Divergence Term in Variational Autoencoders for Text Generation
Victor Prokhorov, Ehsan Shareghi, Yingzhen Li +2
Variational Autoencoders (VAEs) are known to suffer from learning uninformative latent representation of the input due to issues such as approximated posterior collapse, or entangl…
BAND: Biomedical Alert News Dataset
Zihao Fu, Meiru Zhang, Zaiqiao Meng +3
Infectious disease outbreaks continue to pose a significant threat to human health and well-being. To improve disease surveillance and understanding of disease spread, several surv…
Adapting Phrase-based Machine Translation to Normalise Medical Terms in Social Media Messages
Nut Limsopatham, Nigel Collier
Previous studies have shown that health reports in social media, such as DailyStrength and Twitter, have potential for monitoring health conditions (e.g. adverse drug reactions, in…
ReasonGraph: Visualisation of Reasoning Paths
Zongqian Li, Ehsan Shareghi, Nigel Collier
Large Language Models (LLMs) reasoning processes are challenging to analyze due to their complexity and the lack of organized visualization tools. We present ReasonGraph, a web-bas…
COFFEE: A Contrastive Oracle-Free Framework for Event Extraction
Meiru Zhang, Yixuan Su, Zaiqiao Meng +2
Event extraction is a complex information extraction task that involves extracting events from unstructured text. Prior classification-based methods require comprehensive entity an…
500xCompressor: Generalized Prompt Compression for Large Language Models
Zongqian Li, Yixuan Su, Nigel Collier
Prompt compression is crucial for enhancing inference speed, reducing costs, and improving user experience. However, current methods face challenges such as low compression ratios…
Towards a Seamless Integration of Word Senses into Downstream NLP Applications
Mohammad Taher Pilehvar, Jose Camacho-Collados, Roberto Navigli +1
Lexical ambiguity can impede NLP systems from accurate understanding of semantics. Despite its potential benefits, the integration of sense-level information into NLP systems has r…
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders
Fangyu Liu, Ivan VuliÄ, Anna Korhonen +1
Pretrained Masked Language Models (MLMs) have revolutionised NLP in recent years. However, previous work has indicated that off-the-shelf MLMs are not effective as universal lexica…
Prototype-to-Style: Dialogue Generation with Style-Aware Editing on Retrieval Memory
Yixuan Su, Yan Wang, Simon Baker +4
The ability of a dialog system to express prespecified language style during conversations has a direct, positive impact on its usability and on user satisfaction. We introduce a n…
Visual Spatial Reasoning
Fangyu Liu, Guy Emerson, Nigel Collier
Spatial relations are a basic part of human cognition. However, they are expressed in natural language in a variety of ways, and previous work has suggested that current vision-and…
TOAD: Task-Oriented Automatic Dialogs with Diverse Response Styles
Yinhong Liu, Yimai Fang, David Vandyke +1
In light of recent advances in large language models (LLMs), the expectations for the next generation of virtual assistants include enhanced naturalness and adaptability across div…
MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models
Qianchu Liu, Fangyu Liu, Nigel Collier +2
Recent work indicated that pretrained language models (PLMs) such as BERT and RoBERTa can be transformed into effective sentence and word encoders even via simple self-supervised t…
Syndromic classification of Twitter messages
Nigel Collier, Son Doan
Recent studies have shown strong correlation between social networking data and national influenza rates. We expanded upon this success to develop an automated text mining system t…
POSQA: Probe the World Models of LLMs with Size Comparisons
Chang Shu, Jiuzhou Han, Fangyu Liu +2
Embodied language comprehension emphasizes that language understanding is not solely a matter of mental processing in the brain but also involves interactions with the physical and…
Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT
Zaiqiao Meng, Fangyu Liu, Thomas Hikaru Clark +2
Infusing factual knowledge into pre-trained models is fundamental for many knowledge-intensive tasks. In this paper, we proposed Mixture-of-Partitions (MoP), an infusion approach t…
Storage of Natural Language Sentences in a Hopfield Network
Nigel Collier
This paper look at how the Hopfield neural network can be used to store and recall patterns constructed from natural language sentences. As a pattern recognition and storage tool,…
De-Conflated Semantic Representations
Mohammad Taher Pilehvar, Nigel Collier
One major deficiency of most semantic representation techniques is that they usually model a word type as a single point in the semantic space, hence conflating all the meanings th…
LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generations
Caiqi Zhang, Xiaochen Zhu, Chengzu Li +2
Hallucination remains a major challenge for the safe and trustworthy deployment of large language models (LLMs) in factual content generation. Prior work has explored confidence es…
Failure Modes in Multi-Hop QA: The Weakest Link Effect and the Recognition Bottleneck
Meiru Zhang, Zaiqiao Meng, Nigel Collier
Despite scaling to massive context windows, Large Language Models (LLMs) struggle with multi-hop reasoning due to inherent position bias, which causes them to overlook information…
A Pragmatic Guide to Geoparsing Evaluation
Milan Gritta, Mohammad Taher Pilehvar, Nigel Collier
Empirical methods in geoparsing have thus far lacked a standard evaluation framework describing the task, metrics and data used to compare state-of-the-art systems. Evaluation is f…
Towards cross-lingual alerting for bursty epidemic events
Nigel Collier
Background: Online news reports are increasingly becoming a source for event based early warning systems that detect natural disasters. Harnessing the massive volume of information…
Can LLM be a Personalized Judge?
Yijiang River Dong, Tiancheng Hu, Nigel Collier
Ensuring that large language models (LLMs) reflect diverse user values and preferences is crucial as their user bases expand globally. It is therefore encouraging to see the growin…
PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMs
Jiuzhou Han, Nigel Collier, Wray Buntine +1
Large language models (LLMs) have shown great abilities of solving various natural language tasks in different domains. Due to the training objective of LLMs and their pre-training…
What's unusual in online disease outbreak news?
Nigel Collier
Background: Accurate and timely detection of public health events of international concern is necessary to help support risk assessment and response and save lives. Novel event-bas…
Attention Instruction: Amplifying Attention in the Middle via Prompting
Meiru Zhang, Zaiqiao Meng, Nigel Collier
The context window of large language models has been extended to 128k tokens or more. However, language models still suffer from position bias and have difficulty in accessing and…
TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning
Yixuan Su, Fangyu Liu, Zaiqiao Meng +4
Masked language models (MLMs) such as BERT and RoBERTa have revolutionized the field of Natural Language Understanding in the past few years. However, existing pre-trained MLMs oft…
Navigating the Alignment-Calibration Trade-off: A Pareto-Superior Frontier via Model Merging
Tiancheng Hu, Benjamin Minixhofer, Nigel Collier
The "alignment tax" of post-training is typically framed as a drop in task accuracy. We show it also involves a severe loss of calibration, making models overconfident, less reliab…
Quantifying the Persona Effect in LLM Simulations
Tiancheng Hu, Nigel Collier
Large language models (LLMs) have shown remarkable promise in simulating human language and behavior. This study investigates how integrating persona variables-demographic, social,…
Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Models
Yinhong Liu, Zhijiang Guo, Tianya Liang +3
Large Language Models (LLMs) are expected to be predictable and trustworthy to support reliable decision-making systems. Yet current LLMs often show inconsistencies in their judgme…
Generative Language Models Exhibit Social Identity Biases
Tiancheng Hu, Yara Kyrychenko, Steve Rathje +3
The surge in popularity of large language models has given rise to concerns about biases that these models could learn from humans. We investigate whether ingroup solidarity and ou…
Unlocking Structure Measuring: Introducing PDD, an Automatic Metric for Positional Discourse Coherence
Yinhong Liu, Yixuan Su, Ehsan Shareghi +1
Recent large language models (LLMs) have shown remarkable performance in aligning generated text with user intentions across various tasks. When it comes to long-form text generati…
Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models
Zongqian Li, Shaohan Huang, Zewen Chi +5
Modern code generation models exhibit longer outputs, accelerated capability growth, and changed training dynamics, rendering traditional training methodologies, algorithms, and da…
Instruct-SCTG: Guiding Sequential Controlled Text Generation through Instructions
Yinhong Liu, Yixuan Su, Ehsan Shareghi +1
Instruction-tuned large language models have shown remarkable performance in aligning generated text with user intentions across various tasks. However, maintaining human-like disc…
Card-660: Cambridge Rare Word Dataset - a Reliable Benchmark for Infrequent Word Representation Models
Mohammad Taher Pilehvar, Dimitri Kartsaklis, Victor Prokhorov +1
Rare word representation has recently enjoyed a surge of interest, owing to the crucial role that effective handling of infrequent words can play in accurate semantic understanding…
How to tackle an emerging topic? Combining strong and weak labels for Covid news NER
Aleksander Ficek, Fangyu Liu, Nigel Collier
Being able to train Named Entity Recognition (NER) models for emerging topics is crucial for many real-world applications especially in the medical domain where new topics are cont…
Language Models Can See: Plugging Visual Controls in Text Generation
Yixuan Su, Tian Lan, Yahui Liu +5
Generative language models (LMs) such as GPT-2/3 can be prompted to generate text with remarkable quality. While they are designed for text-prompted generation, it remains an open…
COMETA: A Corpus for Medical Entity Linking in the Social Media
Marco Basaldella, Fangyu Liu, Ehsan Shareghi +1
Whilst there has been growing progress in Entity Linking (EL) for general language, existing datasets fail to address the complex nature of health terminology in layman's language.…
Few-Shot Table-to-Text Generation with Prototype Memory
Yixuan Su, Zaiqiao Meng, Simon Baker +1
Neural table-to-text generation models have achieved remarkable progress on an array of tasks. However, due to the data-hungry nature of neural models, their performances strongly…
Visually Grounded Reasoning across Languages and Cultures
Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti +3
The design of widespread vision-and-language datasets and pre-trained encoders directly adopts, or draws inspiration from, the concepts and images of ImageNet. While one can hardly…
Code Is More Than Text: Uncertainty Estimation for Code Generation
Yuling Shi, Caiqi Zhang, Yuexian Li +4
Large language models (LLMs) are increasingly deployed as code generators, where silently wrong programs pose real safety and reliability risks. Reliable uncertainty estimation (UE…
Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective
Huayang Li, Tian Lan, Zihao Fu +5
There are a number of diverging hypotheses about the neural text degeneration problem, i.e., generating repetitive and dull loops, which makes this problem both interesting and con…
LoGU: Long-form Generation with Uncertainty Expressions
Ruihan Yang, Caiqi Zhang, Zhisong Zhang +5
While Large Language Models (LLMs) demonstrate impressive capabilities, they still struggle with generating factually incorrect content (i.e., hallucinations). A promising approach…
FireAct: Toward Language Agent Fine-tuning
Baian Chen, Chang Shu, Ehsan Shareghi +3
Recent efforts have augmented language models (LMs) with external tools or environments, leading to the development of language agents that can reason and act. However, most of the…
Contrastive Search Is What You Need For Neural Text Generation
Yixuan Su, Nigel Collier
Generating text with autoregressive language models (LMs) is of great importance to many natural language processing (NLP) applications. Previous solutions for this task often prod…
Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators
Yinhong Liu, Han Zhou, Zhijiang Guo +4
Large Language Models (LLMs) have demonstrated promising capabilities as automatic evaluators in assessing the quality of generated natural language. However, LLMs still exhibit bi…
On the Effectiveness of Parameter-Efficient Fine-Tuning
Zihao Fu, Haoran Yang, Anthony Man-Cho So +3
Fine-tuning pre-trained models has been ubiquitously proven to be effective in a wide range of NLP tasks. However, fine-tuning the whole model is parameter inefficient as it always…
Visual Pivoting for (Unsupervised) Entity Alignment
Fangyu Liu, Muhao Chen, Dan Roth +1
This work studies the use of visual semantic representations to align entities in heterogeneous knowledge graphs (KGs). Images are natural components of many existing KGs. By combi…
Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder
Zihao Fu, Wai Lam, Qian Yu +4
The sequence-to-sequence (seq2seq) task aims at generating the target sequence based on the given input source sequence. Traditionally, most of the seq2seq task is resolved by the…
A Contrastive Framework for Neural Text Generation
Yixuan Su, Tian Lan, Yan Wang +3
Text generation is of great importance to many natural language processing applications. However, maximization-based decoding methods (e.g. beam search) of neural language models o…
Prompt Compression for Large Language Models: A Survey
Zongqian Li, Yinhong Liu, Yixuan Su +1
Leveraging large language models (LLMs) for complex natural language tasks typically requires long-form prompts to convey detailed requirements and information, which results in in…
Prix-LM: Pretraining for Multilingual Knowledge Base Construction
Wenxuan Zhou, Fangyu Liu, Ivan VuliÄ +2
Knowledge bases (KBs) contain plenty of structured world and commonsense knowledge. As such, they often complement distributional text-based information and facilitate various down…
Conformity in Large Language Models
Xiaochen Zhu, Caiqi Zhang, Tom Stafford +2
The conformity effect describes the tendency of individuals to align their responses with the majority. Studying this bias in large language models (LLMs) is crucial, as LLMs are i…
Time to Revist Exact Match
Auss Abbood, Zaiqiao Meng, Nigel Collier
Temporal question answering is an established method for evaluating temporal reasoning in large language models. Expected answers are often numeric (e.g., dates or durations), yet…
Global Health Monitor: A Web-based System for Detecting and Mapping Infectious Diseases
Son Doan, Quoc-Hung Ngo, Ai Kawazoe +1
We present the Global Health Monitor, an online Web-based system for detecting and mapping infectious disease outbreaks that appear in news stories. The system analyzes English new…
SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors
Tiancheng Hu, Joachim Baumann, Lorenzo Lupo +3
Large language model (LLM) simulations of human behavior have the potential to revolutionize the social and behavioral sciences, if and only if they faithfully reflect real human b…
Plan-then-Generate: Controlled Data-to-Text Generation via Planning
Yixuan Su, David Vandyke, Sihui Wang +2
Recent developments in neural networks have led to the advance in data-to-text generation. However, the lack of ability of neural models to control the structure of generated outpu…
Learning Sparse Sentence Encoding without Supervision: An Exploration of Sparsity in Variational Autoencoders
Victor Prokhorov, Yingzhen Li, Ehsan Shareghi +1
It has been long known that sparsity is an effective inductive bias for learning efficient representation of data in vectors with fixed dimensionality, and it has been explored in…
Biomedical Named Entity Recognition via Dictionary-based Synonym Generalization
Zihao Fu, Yixuan Su, Zaiqiao Meng +1
Biomedical named entity recognition is one of the core tasks in biomedical natural language processing (BioNLP). To tackle this task, numerous supervised/distantly supervised appro…
MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering
Fangyu Liu, Francesco Piccinno, Syrine Krichene +6
Visual language data such as plots, charts, and infographics are ubiquitous in the human world. However, state-of-the-art vision-language models do not perform well on these data.…
When Personalization Meets Reality: A Multi-Faceted Analysis of Personalized Preference Learning
Yijiang River Dong, Tiancheng Hu, Yinhong Liu +2
While Reinforcement Learning from Human Feedback (RLHF) is widely used to align Large Language Models (LLMs) with human preferences, it typically assumes homogeneous preferences ac…
UNCLE: Benchmarking Uncertainty Expressions in Long-Form Generation
Ruihan Yang, Caiqi Zhang, Zhisong Zhang +4
Large Language Models (LLMs) are prone to hallucination, particularly in long-form generations. A promising direction to mitigate hallucination is to teach LLMs to express uncertai…
Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments
Han Zhou, Xingchen Wan, Yinhong Liu +3
Large language models (LLMs) have shown promising abilities as cost-effective and reference-free evaluators for assessing language generation quality. In particular, pairwise LLM e…
An analysis of Twitter messages in the 2011 Tohoku Earthquake
Son Doan, Bao-Khanh Ho Vo, Nigel Collier
Social media such as Facebook and Twitter have proven to be a useful resource to understand public opinion towards real world events. In this paper, we investigate over 1.5 million…
TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law
Zheng Hui, Yijiang River Dong, Ehsan Shareghi +1
As large language models (LLMs) are increasingly deployed in high-risk domains such as law, finance, and medicine, systematically evaluating their domain-specific safety and compli…
When Calibration Rankings Reverse: Accuracy-Controlled Evaluation for Fair Comparison of LLMs
Zhichao Yang, Caiqi Zhang, Ruihan Yang +3
Calibration evaluates whether a model confidence aligns with its empirical accuracy. Existing studies often compare the calibration of different large language models using global…
Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation
Song Liu, Makoto Yamada, Nigel Collier +1
The objective of change-point detection is to discover abrupt property changes lying behind time-series data. In this paper, we present a novel statistical change-point detection a…
Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models
Yupan Huang, Zaiqiao Meng, Fangyu Liu +3
Large language models exhibit enhanced zero-shot performance on various tasks when fine-tuned with instruction-following data. Multimodal instruction-following models extend these…
Dialogue Response Selection with Hierarchical Curriculum Learning
Yixuan Su, Deng Cai, Qingyu Zhou +6
We study the learning of a matching model for dialogue response selection. Motivated by the recent finding that models trained with random negative samples are not ideal in real-wo…
Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models
Ej Zhou, Caiqi Zhang, Tiancheng Hu +4
Confidence calibration, the alignment of a model's predicted confidence with its actual accuracy, is crucial for the reliable deployment of Large Language Models (LLMs). However, t…
A Stability Analysis of Fine-Tuning a Pre-Trained Model
Zihao Fu, Anthony Man-Cho So, Nigel Collier
Fine-tuning a pre-trained model (such as BERT, ALBERT, RoBERTa, T5, GPT, etc.) has proven to be one of the most promising paradigms in recent NLP research. However, numerous recent…
Generating Knowledge Graph Paths from Textual Definitions using Sequence-to-Sequence Models
Victor Prokhorov, Mohammad Taher Pilehvar, Nigel Collier
We present a novel method for mapping unrestricted text to knowledge graph entities by framing the task as a sequence-to-sequence problem. Specifically, given the encoded state of…
TopViewRS: Vision-Language Models as Top-View Spatial Reasoners
Chengzu Li, Caiqi Zhang, Han Zhou +3
Top-view perspective denotes a typical way in which humans read and reason over different types of maps, and it is vital for localization and navigation of humans as well as of `no…
Atomic Calibration of LLMs in Long-Form Generations
Caiqi Zhang, Ruihan Yang, Zhisong Zhang +4
Large language models (LLMs) often suffer from hallucinations, posing significant challenges for real-world applications. Confidence calibration, as an effective indicator of hallu…
Improving Word Translation via Two-Stage Contrastive Learning
Yaoyiran Li, Fangyu Liu, Nigel Collier +2
Word translation or bilingual lexicon induction (BLI) is a key cross-lingual task, aiming to bridge the lexical gap between different languages. In this work, we propose a robust a…
LUQ: Long-text Uncertainty Quantification for LLMs
Caiqi Zhang, Fangyu Liu, Marco Basaldella +1
Large Language Models (LLMs) have demonstrated remarkable capability in a variety of NLP tasks. However, LLMs are also prone to generate nonfactual content. Uncertainty Quantificat…
Demystifying Multi-Agent Debate: The Role of Confidence and Diversity
Xiaochen Zhu, Caiqi Zhang, Yizhou Chi +3
Multi-agent debate (MAD) is widely used to improve large language model (LLM) performance through test-time scaling, yet recent work shows that vanilla MAD often underperforms simp…
Mapping Text to Knowledge Graph Entities using Multi-Sense LSTMs
Dimitri Kartsaklis, Mohammad Taher Pilehvar, Nigel Collier
This paper addresses the problem of mapping natural language text to knowledge base entities. The mapping process is approached as a composition of a phrase or a sentence into a po…
Steer Model beyond Assistant: Controlling System Prompt Strength via Contrastive Decoding
Yijiang River Dong, Tiancheng Hu, Zheng Hui +1
Large language models excel at complex instructions yet struggle to deviate from their helpful assistant persona, as post-training instills strong priors that resist conflicting in…
DePlot: One-shot visual language reasoning by plot-to-table translation
Fangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno +7
Visual language such as charts and plots is ubiquitous in the human world. Comprehending plots and charts requires strong reasoning skills. Prior state-of-the-art (SOTA) models req…
Privacy-R1: Privacy-Aware Multi-LLM Agent Collaboration via Reinforcement Learning
Zheng Hui, Yijiang River Dong, Sanhanat Sivapiromrat +2
When users submit queries to Large Language Models (LLMs), their prompts can often contain sensitive data, forcing a difficult choice: Send the query to a powerful proprietary LLM…
Value of Information: A Framework for Human-Agent Communication
Yijiang River Dong, Tiancheng Hu, Zheng Hui +4
Large Language Model (LLM) agents deployed for real-world tasks face a fundamental dilemma: user requests are underspecified, yet agents must decide whether to act on incomplete in…
All Roads Lead to Rome: Graph-Based Confidence Estimation for Large Language Model Reasoning
Caiqi Zhang, Chang Shu, Ehsan Shareghi +1
Confidence estimation is essential for the reliable deployment of large language models (LLMs). Existing methods are primarily designed for factual QA tasks and often fail to gener…