26 citations · 150 across the 47 of their papers we have counts for
36 papers · 1 filter
The Powerless Noise: How Experimental Settings Shape the Reported Power of Noise
Michał Mazuryk, Fleur Dolmans, Louis Gehringer +3
Recent work has suggested that adding irrelevant documents to the input of retrieval-augmented generation (RAG) systems can improve question-answering performance, a phenomenon ref…
Improving the Efficiency and Effectiveness of LLM Knowledge Distillation for Conversational Search
Stan Fris, Jan Hutter, Jan Henrik Bertrand +2
Conversational Search (CS) considers retrieval of relevant documents based on conversational context. Large Language Models (LLMs) have significantly enhanced CS by enabling effect…
Hypencoder Revisited: Reproducibility and Analysis of Non-Linear Scoring for First-Stage Retrieval
Arne Eichholtz, Yongkang Li, Jutte Vijverberg +2
The Hypencoder, proposed by Killingback et al., is a retrieval framework that replaces the fixed inner-product scoring function used in standard bi-encoders with a query-specific n…
Total Recall QA: A Verifiable Evaluation Suite for Deep Research Agents
Mahta Rafiee, Heydar Soudani, Zahra Abbasiantaeb +3
Deep research agents have emerged as LLM-based systems designed to perform multi-step information seeking and reasoning over large, open-domain sources to answer complex questions…
Re-Rankers as Relevance Judges
Chuan Meng, Jiqun Liu, Mohammad Aliannejadi +3
Using large language models (LLMs) to predict relevance judgments has shown promising results. Most studies treat this task as a distinct research line, e.g., focusing on prompt de…
Investigating LLM Variability in Personalized Conversational Information Retrieval
Simon Lupart, Daniël van Dijk, Eric Langezaal +2
Personalized Conversational Information Retrieval (CIR) has seen rapid progress in recent years, driven by the development of Large Language Models (LLMs). Personalized CIR aims to…
A Reproducibility Study of Product-side Fairness in Bundle Recommendation
Huy-Son Nguyen, Yuanna Liu, Masoud Mansoury +3
Recommender systems are known to exhibit fairness issues, particularly on the product side, where products and their associated suppliers receive unequal exposure in recommended re…
Conversational Search: From Fundamentals to Frontiers in the LLM Era
Fengran Mo, Chuan Meng, Mohammad Aliannejadi +1
Conversational search enables multi-turn interactions between users and systems to fulfill users' complex information needs. During this interaction, the system should understand t…
Understanding Visual Saliency of Outlier Items in Product Search
Fatemeh Sarvi, Mohammad Aliannejadi, Sebastian Schelter +1
In two-sided marketplaces, items compete for user attention, which translates to revenue for suppliers. Item exposure, indicated by the amount of attention items receive in a ranki…
Conversational Gold: Evaluating Personalized Conversational Search System using Gold Nuggets
Zahra Abbasiantaeb, Simon Lupart, Leif Azzopardi +2
The rise of personalized conversational search systems has been driven by advancements in Large Language Models (LLMs), enabling these systems to retrieve and generate answers for…
Improving the Reusability of Conversational Search Test Collections
Zahra Abbasiantaeb, Chuan Meng, Leif Azzopardi +1
Incomplete relevance judgments limit the reusability of test collections. When new systems are compared to previous systems that contributed to the pool, they often face a disadvan…
Judging the Judges: A Collection of LLM-Generated Relevance Judgements
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi +6
Using Large Language Models (LLMs) for relevance assessments offers promising opportunities to improve Information Retrieval (IR), Natural Language Processing (NLP), and related fi…
Multi-Turn Multi-Modal Question Clarification for Enhanced Conversational Understanding
Kimia Ramezan, Alireza Amiri Bavandpour, Yifei Yuan +2
Conversational query clarification enables users to refine their search queries through interactive dialogue, improving search effectiveness. Traditional approaches rely on text-ba…
Reproducing NevIR: Negation in Neural Information Retrieval
Coen van den Elsen, Francien Barkhof, Thijmen Nijdam +2
Negation is a fundamental aspect of human communication, yet it remains a challenge for Language Models (LMs) in Information Retrieval (IR). Despite the heavy reliance of modern ne…
Repeat-bias-aware Optimization of Beyond-accuracy Metrics for Next Basket Recommendation
Yuanna Liu, Ming Li, Mohammad Aliannejadi +1
In next basket recommendation (NBR) a set of items is recommended to users based on their historical basket sequences. In many domains, the recommended baskets consist of both repe…
IRLab@iKAT24: Learned Sparse Retrieval with Multi-aspect LLM Query Generation for Conversational Search
Simon Lupart, Zahra Abbasiantaeb, Mohammad Aliannejadi
The Interactive Knowledge Assistant Track (iKAT) 2024 focuses on advancing conversational assistants, able to adapt their interaction and responses from personalized user knowledge…
DiSCo: LLM Knowledge Distillation for Efficient Sparse Retrieval in Conversational Search
Simon Lupart, Mohammad Aliannejadi, Evangelos Kanoulas
Conversational Search (CS) involves retrieving relevant documents from a corpus while considering the conversational context, integrating retrieval with context modeling. Recent ad…
A Comparative Analysis of Faithfulness Metrics and Humans in Citation Evaluation
Weijia Zhang, Mohammad Aliannejadi, Jiahuan Pei +3
Large language models (LLMs) often generate content with unsupported or unverifiable content, known as "hallucinations." To address this, retrieval-augmented LLMs are employed to i…
LLMJudge: LLMs for Relevance Judgments
Hossein A. Rahmani, Emine Yilmaz, Nick Craswell +6
The LLMJudge challenge is organized as part of the LLM4Eval workshop at SIGIR 2024. Test collections are essential for evaluating information retrieval (IR) systems. The evaluation…
Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi +6
The first edition of the workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) took place in July 2024, co-located with the ACM SIGIR Conference…
Generative Retrieval with Few-shot Indexing
Arian Askari, Chuan Meng, Mohammad Aliannejadi +3
Existing generative retrieval (GR) methods rely on training-based indexing, which fine-tunes a model to memorise associations between queries and the document identifiers (docids)…
Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics
Weijia Zhang, Mohammad Aliannejadi, Yifei Yuan +3
Large language models (LLMs) often produce unsupported or unverifiable content, known as "hallucinations." To mitigate this, retrieval-augmented LLMs incorporate citations, groundi…
Can We Use Large Language Models to Fill Relevance Judgment Holes?
Zahra Abbasiantaeb, Chuan Meng, Leif Azzopardi +1
Incomplete relevance judgments limit the re-usability of test collections. When new systems are compared against previous systems used to build the pool of judged documents, they o…
TREC iKAT 2023: A Test Collection for Evaluating Conversational and Interactive Knowledge Assistants
Mohammad Aliannejadi, Zahra Abbasiantaeb, Shubham Chatterjee +2
Conversational information seeking has evolved rapidly in the last few years with the development of Large Language Models (LLMs), providing the basis for interpreting and respondi…