6 citations · 17 across the 20 of their papers we have counts for
20 papers
Uncertainty-based Debiasing and Unlearning for Decontamination
Guangzhi Sun, Xiao Zhan, Mark Gales
Benchmark-based evaluation is the dominant paradigm for assessing large language model (LLM) capabilities, yet data contamination inflates reported performance and undermines fair…
SkillAggregation: Reference-free LLM-Dependent Aggregation
Guangzhi Sun, Anmol Kagrecha, Potsawee Manakul +2
Large Language Models (LLMs) are increasingly used to assess NLP tasks due to their ability to generate human-like judgments. Single LLMs were used initially, however, recent work…
Finetuning LLMs for Comparative Assessment Tasks
Vatsal Raina, Adian Liusie, Mark Gales
Automated assessment in natural language generation is a challenging task. Instruction-tuned large language models (LLMs) have shown promise in reference-free evaluation, particula…
Controlling Whisper: Universal Acoustic Adversarial Attacks to Control Speech Foundation Models
Vyas Raina, Mark Gales
Speech enabled foundation models, either in the form of flexible speech recognition based systems or audio-prompted large language models (LLMs), are becoming increasingly popular.…
Cross-Lingual Transfer Learning for Speech Translation
Rao Ma, Mengjie Qian, Yassir Fathullah +3
There has been increasing interest in building multilingual foundation models for NLP and speech research. This paper examines how to expand the speech translation capability of th…
CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models
Guangzhi Sun, Potsawee Manakul, Adian Liusie +4
Multimodal foundation models are prone to hallucination, generating outputs that either contradict the input or are not grounded by factual information. Given the diversity in arch…
Question-Based Retrieval using Atomic Units for Enterprise RAG
Vatsal Raina, Mark Gales
Enterprise retrieval augmented generation (RAG) offers a highly flexible framework for combining powerful large language models (LLMs) with internal, possibly temporally changing,…
Muting Whisper: A Universal Acoustic Adversarial Attack on Speech Foundation Models
Vyas Raina, Rao Ma, Charles McGhee +2
Recent developments in large speech foundation models like Whisper have led to their widespread use in many automatic speech recognition (ASR) applications. These systems incorpora…
Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise Comparisons
Adian Liusie, Vatsal Raina, Yassir Fathullah +1
LLM-as-a-judge approaches are a practical and effective way of assessing a range of text tasks. However, when using pairwise comparisons to rank a set of candidates, the computatio…
Question Difficulty Ranking for Multiple-Choice Reading Comprehension
Vatsal Raina, Mark Gales
Multiple-choice (MC) tests are an efficient method to assess English learners. It is useful for test creators to rank candidate MC questions by difficulty during exam curation. Typ…
WaterJudge: Quality-Detection Trade-off when Watermarking Large Language Models
Piotr Molenda, Adian Liusie, Mark J. F. Gales
Watermarking generative-AI systems, such as LLMs, has gained considerable interest, driven by their enhanced capabilities across a wide range of tasks. Although current approaches…
Teacher-Student Training for Debiasing: General Permutation Debiasing for Large Language Models
Adian Liusie, Yassir Fathullah, Mark J. F. Gales
Large Language Models (LLMs) have demonstrated impressive zero-shot capabilities and versatility in NLP tasks, however they sometimes fail to maintain crucial invariances for speci…
LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History
Akash Gupta, Ivaxi Sheth, Vyas Raina +2
With the recent emergence of powerful instruction-tuned large language models (LLMs), various helpful conversational Artificial Intelligence (AI) systems have been deployed across…
Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment
Vyas Raina, Adian Liusie, Mark Gales
Large Language Models (LLMs) are powerful zero-shot assessors used in real-world situations such as assessing written exams and benchmarking systems. Despite these critical applica…
An Information-Theoretic Approach to Analyze NLP Classification Tasks
Luran Wang, Mark Gales, Vatsal Raina
Understanding the importance of the inputs on the output is useful across many tasks. This work provides an information-theoretic framework to analyse the influence of inputs for t…
Assessing Distractors in Multiple-Choice Tests
Vatsal Raina, Adian Liusie, Mark Gales
Multiple-choice tests are a common approach for assessing candidates' comprehension skills. Standard multiple-choice reading comprehension exams require candidates to select the co…
Structural-Based Uncertainty in Deep Learning Across Anatomical Scales: Analysis in White Matter Lesion Segmentation
Nataliia Molchanova, Vatsal Raina, Andrey Malinin +7
This paper explores uncertainty quantification (UQ) as an indicator of the trustworthiness of automated deep-learning (DL) tools in the context of white matter lesion (WML) segment…
Minimum Bayes' Risk Decoding for System Combination of Grammatical Error Correction Systems
Vyas Raina, Mark Gales
For sequence-to-sequence tasks it is challenging to combine individual system outputs. Further, there is also often a mismatch between the decoding criterion and the one used for a…
Is it Possible to Modify Text to a Target Readability Level? An Initial Investigation Using Zero-Shot Large Language Models
Asma Farajidizaji, Vatsal Raina, Mark Gales
Text simplification is a common task where the text is adapted to make it easier to understand. Similarly, text elaboration can make a passage more sophisticated, offering a method…
Who Needs Decoders? Efficient Estimation of Sequence-level Attributes
Yassir Fathullah, Puria Radmard, Adian Liusie +1
State-of-the-art sequence-to-sequence models often require autoregressive decoding, which can be highly expensive. However, for some downstream tasks such as out-of-distribution (O…