18 citations · 52 across the 20 of their papers we have counts for
5 papers · 1 filter
Finetuning LLMs for Comparative Assessment Tasks
Vatsal Raina, Adian Liusie, Mark Gales
Automated assessment in natural language generation is a challenging task. Instruction-tuned large language models (LLMs) have shown promise in reference-free evaluation, particula…
Question-Based Retrieval using Atomic Units for Enterprise RAG
Vatsal Raina, Mark Gales
Enterprise retrieval augmented generation (RAG) offers a highly flexible framework for combining powerful large language models (LLMs) with internal, possibly temporally changing,…
Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise Comparisons
Adian Liusie, Vatsal Raina, Yassir Fathullah +1
LLM-as-a-judge approaches are a practical and effective way of assessing a range of text tasks. However, when using pairwise comparisons to rank a set of candidates, the computatio…
Question Difficulty Ranking for Multiple-Choice Reading Comprehension
Vatsal Raina, Mark Gales
Multiple-choice (MC) tests are an efficient method to assess English learners. It is useful for test creators to rank candidate MC questions by difficulty during exam curation. Typ…
An Information-Theoretic Approach to Analyze NLP Classification Tasks
Luran Wang, Mark Gales, Vatsal Raina
Understanding the importance of the inputs on the output is useful across many tasks. This work provides an information-theoretic framework to analyse the influence of inputs for t…