3 citations · 10 across the 18 of their papers we have counts for
7 papers · 1 filter
Self-Correcting Large Language Models: Generation vs. Multiple Choice
Hossein A. Rahmani, Satyapriya Krishna, Xi Wang +2
Large language models have recently demonstrated remarkable abilities to self-correct their responses through iterative refinement, often referred to as self-consistency or self-re…
Beyond Internal Data: Bounding and Estimating Fairness from Incomplete Data
Varsha Ramineni, Hossein A. Rahmani, Emine Yilmaz +1
Ensuring fairness in AI systems is critical, especially in high-stakes domains such as lending, hiring, and healthcare. This urgency is reflected in emerging global regulations tha…
PREF: Reference-Free Evaluation of Personalised Text Generation in LLMs
Xiao Fu, Hossein A. Rahmani, Bin Wu +3
Personalised text generation is essential for user-centric information systems, yet most evaluation methods overlook the individuality of users. We introduce \textbf{PREF}, a \text…
Beyond Internal Data: Constructing Complete Datasets for Fairness Testing
Varsha Ramineni, Hossein A. Rahmani, Emine Yilmaz +1
As AI becomes prevalent in high-risk domains and decision-making, it is essential to test for potential harms and biases. This urgency is reflected by the global emergence of AI re…
Overview of the TREC 2023 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +5
This is the fifth year of the TREC Deep Learning track. As in previous years, we leverage the MS MARCO datasets that made hundreds of thousands of human-annotated training labels a…
Towards Understanding Bias in Synthetic Data for Evaluation
Hossein A. Rahmani, Varsha Ramineni, Emine Yilmaz +2
Test collections are crucial for evaluating Information Retrieval (IR) systems. Creating a diverse set of user queries for these collections can be challenging, and obtaining relev…
Judging the Judges: A Collection of LLM-Generated Relevance Judgements
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi +6
Using Large Language Models (LLMs) for relevance assessments offers promising opportunities to improve Information Retrieval (IR), Natural Language Processing (NLP), and related fi…