39 citations · 62 across the 19 of their papers we have counts for
21 papers
Embedding the Teacher: Distilling vLLM Preferences for Scalable Image Retrieval
Eric He, Akash Gupta, Adian Liusie +4
Text--image retrieval is necessary for applications such as product recommendation. Embedding-based approaches like CLIP enable efficient large-scale retrieval via vector similarit…
Finetuning LLMs for Comparative Assessment Tasks
Vatsal Raina, Adian Liusie, Mark Gales
Automated assessment in natural language generation is a challenging task. Instruction-tuned large language models (LLMs) have shown promise in reference-free evaluation, particula…
CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models
Guangzhi Sun, Potsawee Manakul, Adian Liusie +4
Multimodal foundation models are prone to hallucination, generating outputs that either contradict the input or are not grounded by factual information. Given the diversity in arch…
Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise Comparisons
Adian Liusie, Vatsal Raina, Yassir Fathullah +1
LLM-as-a-judge approaches are a practical and effective way of assessing a range of text tasks. However, when using pairwise comparisons to rank a set of candidates, the computatio…
WaterJudge: Quality-Detection Trade-off when Watermarking Large Language Models
Piotr Molenda, Adian Liusie, Mark J. F. Gales
Watermarking generative-AI systems, such as LLMs, has gained considerable interest, driven by their enhanced capabilities across a wide range of tasks. Although current approaches…
Teacher-Student Training for Debiasing: General Permutation Debiasing for Large Language Models
Adian Liusie, Yassir Fathullah, Mark J. F. Gales
Large Language Models (LLMs) have demonstrated impressive zero-shot capabilities and versatility in NLP tasks, however they sometimes fail to maintain crucial invariances for speci…
Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment
Vyas Raina, Adian Liusie, Mark Gales
Large Language Models (LLMs) are powerful zero-shot assessors used in real-world situations such as assessing written exams and benchmarking systems. Despite these critical applica…
Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM
Xiaoding Lu, Zongyi Liu, Adian Liusie +4
In conversational AI research, there's a noticeable trend towards developing models with a larger number of parameters, exemplified by models like ChatGPT. While these expansive mo…
Assessing Distractors in Multiple-Choice Tests
Vatsal Raina, Adian Liusie, Mark Gales
Multiple-choice tests are a common approach for assessing candidates' comprehension skills. Standard multiple-choice reading comprehension exams require candidates to select the co…
Investigating the Emergent Audio Classification Ability of ASR Foundation Models
Rao Ma, Adian Liusie, Mark J. F. Gales +1
Text and vision foundation models can perform many tasks in a zero-shot setting, a desirable property that enables these systems to be applied in general and low-resource settings.…
Mitigating Word Bias in Zero-shot Prompt-based Classifiers
Adian Liusie, Potsawee Manakul, Mark J. F. Gales
Prompt-based classifiers are an attractive approach for zero-shot classification. However, the precise choice of the prompt template and label words can largely influence performan…
Zero-shot Audio Topic Reranking using Large Language Models
Mengjie Qian, Rao Ma, Adian Liusie +3
Multimodal Video Search by Examples (MVSE) investigates using video clips as the query term for information retrieval, rather than the more traditional text query. This enables far…
Analyzing Multiple-Choice Reading and Listening Comprehension Tests
Vatsal Raina, Adian Liusie, Mark Gales
Multiple-choice reading and listening comprehension tests are an important part of language assessment. Content creators for standard educational tests need to carefully curate que…
LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models
Adian Liusie, Potsawee Manakul, Mark J. F. Gales
Current developments in large language models (LLMs) have enabled impressive zero-shot capabilities across various natural language tasks. An interesting application of these syste…
CUED at ProbSum 2023: Hierarchical Ensemble of Summarization Models
Potsawee Manakul, Yassir Fathullah, Adian Liusie +3
In this paper, we consider the challenge of summarizing patients' medical progress notes in a limited data setting. For the Problem List Summarization (shared task 1A) at the BioNL…
Analysis of the Cambridge Multiple-Choice Questions Reading Dataset with a Focus on Candidate Response Distribution
Adian Liusie, Vatsal Raina, Andrew Mullooly +2
Multiple choice exams are widely used to assess candidates across a diverse range of domains and tasks. To moderate question quality, newly proposed questions often pass through pr…
Who Needs Decoders? Efficient Estimation of Sequence-level Attributes
Yassir Fathullah, Puria Radmard, Adian Liusie +1
State-of-the-art sequence-to-sequence models often require autoregressive decoding, which can be highly expensive. However, for some downstream tasks such as out-of-distribution (O…
Rewarding Chatbots for Real-World Engagement with Millions of Users
Robert Irvine, Douglas Boubert, Vyas Raina +11
The emergence of pretrained large language models has led to the deployment of a range of social chatbots for chitchat. Although these chatbots demonstrate language ability and flu…
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Potsawee Manakul, Adian Liusie, Mark J. F. Gales
Generative Large Language Models (LLMs) such as GPT-3 are capable of generating highly fluent responses to a wide variety of user prompts. However, LLMs are known to hallucinate fa…
MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarization
Potsawee Manakul, Adian Liusie, Mark J. F. Gales
State-of-the-art summarization systems can generate highly fluent summaries. These summaries, however, may contain factual inconsistencies and/or information not present in the sou…
World Knowledge in Multiple Choice Reading Comprehension
Adian Liusie, Vatsal Raina, Mark Gales
Recently it has been shown that without any access to the contextual passage, multiple choice reading comprehension (MCRC) systems are able to answer questions significantly better…