26 citations · 150 across the 47 of their papers we have counts for
50 papers
Prompting Strategy Effects Depend on the Judge: A Multilingual Study of LLM-as-a-Judge Robustness
Asal Mehrabi, Mohammad Aliannejadi, Clemencia Siro
LLM-as-a-Judge evaluation is increasingly applied to multilingual tasks, yet prompting strategies such as Chain-of-Thought and Role Conditioning are recommended based solely on Eng…
The Powerless Noise: How Experimental Settings Shape the Reported Power of Noise
Michał Mazuryk, Fleur Dolmans, Louis Gehringer +3
Recent work has suggested that adding irrelevant documents to the input of retrieval-augmented generation (RAG) systems can improve question-answering performance, a phenomenon ref…
Improving the Efficiency and Effectiveness of LLM Knowledge Distillation for Conversational Search
Stan Fris, Jan Hutter, Jan Henrik Bertrand +2
Conversational Search (CS) considers retrieval of relevant documents based on conversational context. Large Language Models (LLMs) have significantly enhanced CS by enabling effect…
Hypencoder Revisited: Reproducibility and Analysis of Non-Linear Scoring for First-Stage Retrieval
Arne Eichholtz, Yongkang Li, Jutte Vijverberg +2
The Hypencoder, proposed by Killingback et al., is a retrieval framework that replaces the fixed inner-product scoring function used in standard bi-encoders with a query-specific n…
Total Recall QA: A Verifiable Evaluation Suite for Deep Research Agents
Mahta Rafiee, Heydar Soudani, Zahra Abbasiantaeb +3
Deep research agents have emerged as LLM-based systems designed to perform multi-step information seeking and reasoning over large, open-domain sources to answer complex questions…
Do Images Clarify? A Study on the Effect of Images on Clarifying Questions in Conversational Search
Clemencia Siro, Zahra Abbasiantaeb, Yifei Yuan +2
Conversational search systems increasingly employ clarifying questions to refine user queries and improve the search experience. Previous studies have demonstrated the usefulness o…
Learning to Judge: LLMs Designing and Applying Evaluation Rubrics
Clemencia Siro, Pourya Aliannejadi, Mohammad Aliannejadi
Large language models (LLMs) are increasingly used as evaluators for natural language generation, applying human-defined rubrics to assess system outputs. However, human rubrics ar…
Re-Rankers as Relevance Judges
Chuan Meng, Jiqun Liu, Mohammad Aliannejadi +3
Using large language models (LLMs) to predict relevance judgments has shown promising results. Most studies treat this task as a distinct research line, e.g., focusing on prompt de…
Investigating LLM Variability in Personalized Conversational Information Retrieval
Simon Lupart, Daniël van Dijk, Eric Langezaal +2
Personalized Conversational Information Retrieval (CIR) has seen rapid progress in recent years, driven by the development of Large Language Models (LLMs). Personalized CIR aims to…
ChatR1: Reinforcement Learning for Conversational Reasoning and Retrieval Augmented Question Answering
Simon Lupart, Mohammad Aliannejadi, Evangelos Kanoulas
We present ChatR1, a reasoning framework based on reinforcement learning (RL) for conversational question answering (CQA). Reasoning plays an important role in CQA, where user inte…
Summarize-Exemplify-Reflect: Data-driven Insight Distillation Empowers LLMs for Few-shot Tabular Classification
Yifei Yuan, Jiatong Li, Weijia Zhang +3
Recent studies show the promise of large language models (LLMs) for few-shot tabular classification but highlight challenges due to the variability in structured data. To address t…
A Reproducibility Study of Product-side Fairness in Bundle Recommendation
Huy-Son Nguyen, Yuanna Liu, Masoud Mansoury +3
Recommender systems are known to exhibit fairness issues, particularly on the product side, where products and their associated suppliers receive unequal exposure in recommended re…
Conversational Search: From Fundamentals to Frontiers in the LLM Era
Fengran Mo, Chuan Meng, Mohammad Aliannejadi +1
Conversational search enables multi-turn interactions between users and systems to fulfill users' complex information needs. During this interaction, the system should understand t…
RAVENEA: A Benchmark for Multimodal Retrieval-Augmented Visual Culture Understanding
Jiaang Li, Yifei Yuan, Wenyan Li +8
As vision-language models (VLMs) become increasingly integrated into daily life, the need for accurate visual culture understanding is becoming critical. Yet, these models frequent…
Query Understanding in LLM-based Conversational Information Seeking
Yifei Yuan, Zahra Abbasiantaeb, Yang Deng +1
Query understanding in Conversational Information Seeking (CIS) involves accurately interpreting user intent through context-aware interactions. This includes resolving ambiguities…
Understanding Visual Saliency of Outlier Items in Product Search
Fatemeh Sarvi, Mohammad Aliannejadi, Sebastian Schelter +1
In two-sided marketplaces, items compete for user attention, which translates to revenue for suppliers. Item exposure, indicated by the amount of attention items receive in a ranki…
Conversational Gold: Evaluating Personalized Conversational Search System using Gold Nuggets
Zahra Abbasiantaeb, Simon Lupart, Leif Azzopardi +2
The rise of personalized conversational search systems has been driven by advancements in Large Language Models (LLMs), enabling these systems to retrieve and generate answers for…
Improving the Reusability of Conversational Search Test Collections
Zahra Abbasiantaeb, Chuan Meng, Leif Azzopardi +1
Incomplete relevance judgments limit the reusability of test collections. When new systems are compared to previous systems that contributed to the pool, they often face a disadvan…
Judging the Judges: A Collection of LLM-Generated Relevance Judgements
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi +6
Using Large Language Models (LLMs) for relevance assessments offers promising opportunities to improve Information Retrieval (IR), Natural Language Processing (NLP), and related fi…
Multi-Turn Multi-Modal Question Clarification for Enhanced Conversational Understanding
Kimia Ramezan, Alireza Amiri Bavandpour, Yifei Yuan +2
Conversational query clarification enables users to refine their search queries through interactive dialogue, improving search effectiveness. Traditional approaches rely on text-ba…
PSCon: Product Search Through Conversations
Jie Zou, Mohammad Aliannejadi, Evangelos Kanoulas +5
Conversational Product Search ( CPS ) systems interact with users via natural language to offer personalized and context-aware product lists. However, most existing research on CPS…
Reproducing NevIR: Negation in Neural Information Retrieval
Coen van den Elsen, Francien Barkhof, Thijmen Nijdam +2
Negation is a fundamental aspect of human communication, yet it remains a challenge for Language Models (LMs) in Information Retrieval (IR). Despite the heavy reliance of modern ne…
Repeat-bias-aware Optimization of Beyond-accuracy Metrics for Next Basket Recommendation
Yuanna Liu, Ming Li, Mohammad Aliannejadi +1
In next basket recommendation (NBR) a set of items is recommended to users based on their historical basket sequences. In many domains, the recommended baskets consist of both repe…
IRLab@iKAT24: Learned Sparse Retrieval with Multi-aspect LLM Query Generation for Conversational Search
Simon Lupart, Zahra Abbasiantaeb, Mohammad Aliannejadi
The Interactive Knowledge Assistant Track (iKAT) 2024 focuses on advancing conversational assistants, able to adapt their interaction and responses from personalized user knowledge…