26 citations · 150 across the 47 of their papers we have counts for
21 papers · 1 filter
IRLab@iKAT24: Learned Sparse Retrieval with Multi-aspect LLM Query Generation for Conversational Search
Simon Lupart, Zahra Abbasiantaeb, Mohammad Aliannejadi
The Interactive Knowledge Assistant Track (iKAT) 2024 focuses on advancing conversational assistants, able to adapt their interaction and responses from personalized user knowledge…
AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMs
Clemencia Siro, Yifei Yuan, Mohammad Aliannejadi +1
Generating diverse and effective clarifying questions is crucial for improving query understanding and retrieval performance in open-domain conversational search (CS) systems. We p…
DiSCo: LLM Knowledge Distillation for Efficient Sparse Retrieval in Conversational Search
Simon Lupart, Mohammad Aliannejadi, Evangelos Kanoulas
Conversational Search (CS) involves retrieving relevant documents from a corpus while considering the conversational context, integrating retrieval with context modeling. Recent ad…
Unlocking Markets: A Multilingual Benchmark to Cross-Market Question Answering
Yifei Yuan, Yang Deng, Anders Søgaard +1
Users post numerous product-related questions on e-commerce platforms, affecting their purchase decisions. Product-related question answering (PQA) entails utilizing product-relate…
A Comparative Analysis of Faithfulness Metrics and Humans in Citation Evaluation
Weijia Zhang, Mohammad Aliannejadi, Jiahuan Pei +3
Large language models (LLMs) often generate content with unsupported or unverifiable content, known as "hallucinations." To address this, retrieval-augmented LLMs are employed to i…
LLMJudge: LLMs for Relevance Judgments
Hossein A. Rahmani, Emine Yilmaz, Nick Craswell +6
The LLMJudge challenge is organized as part of the LLM4Eval workshop at SIGIR 2024. Test collections are essential for evaluating information retrieval (IR) systems. The evaluation…
Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi +6
The first edition of the workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) took place in July 2024, co-located with the ACM SIGIR Conference…
Generative Retrieval with Few-shot Indexing
Arian Askari, Chuan Meng, Mohammad Aliannejadi +3
Existing generative retrieval (GR) methods rely on training-based indexing, which fine-tunes a model to memorise associations between queries and the document identifiers (docids)…
Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics
Weijia Zhang, Mohammad Aliannejadi, Yifei Yuan +3
Large language models (LLMs) often produce unsupported or unverifiable content, known as "hallucinations." To mitigate this, retrieval-augmented LLMs incorporate citations, groundi…
Can We Use Large Language Models to Fill Relevance Judgment Holes?
Zahra Abbasiantaeb, Chuan Meng, Leif Azzopardi +1
Incomplete relevance judgments limit the re-usability of test collections. When new systems are compared against previous systems used to build the pool of judged documents, they o…
TREC iKAT 2023: A Test Collection for Evaluating Conversational and Interactive Knowledge Assistants
Mohammad Aliannejadi, Zahra Abbasiantaeb, Shubham Chatterjee +2
Conversational information seeking has evolved rapidly in the last few years with the development of Large Language Models (LLMs), providing the basis for interpreting and respondi…
Are We Really Achieving Better Beyond-Accuracy Performance in Next Basket Recommendation?
Ming Li, Yuanna Liu, Sami Jullien +4
Next basket recommendation (NBR) is a special type of sequential recommendation that is increasingly receiving attention. So far, most NBR studies have focused on optimizing the ac…
Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs
Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke
In ad-hoc retrieval, evaluation relies heavily on user actions, including implicit feedback. In a conversational setting such signals are usually unavailable due to the nature of t…
Ranked List Truncation for Large Language Model-based Re-Ranking
Chuan Meng, Negar Arabzadeh, Arian Askari +2
We study ranked list truncation (RLT) from a novel "retrieve-then-re-rank" perspective, where we optimize re-ranking by truncating the retrieved list (i.e., trim re-ranking candida…
Context Does Matter: Implications for Crowdsourced Evaluation Labels in Task-Oriented Dialogue Systems
Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke
Crowdsourced labels play a crucial role in evaluating task-oriented dialogue systems (TDSs). Obtaining high-quality and consistent ground-truth labels from annotators presents chal…
Query Performance Prediction using Relevance Judgments Generated by Large Language Models
Chuan Meng, Negar Arabzadeh, Arian Askari +2
Query performance prediction (QPP) aims to estimate the retrieval quality of a search system for a query without human relevance judgments. Previous QPP methods typically return a…
Generating Multi-Aspect Queries for Conversational Search
Zahra Abbasiantaeb, Simon Lupart, Mohammad Aliannejadi
Conversational information seeking (CIS) systems aim to model the user's information need within the conversational context and retrieve the relevant information. One major approac…
CAUSE: Counterfactual Assessment of User Satisfaction Estimation in Task-Oriented Dialogue Systems
Amin Abolghasemi, Zhaochun Ren, Arian Askari +3
An important unexplored aspect in previous work on user satisfaction estimation for Task-Oriented Dialogue (TOD) systems is their evaluation in terms of robustness for the identifi…
TREC iKAT 2023: The Interactive Knowledge Assistance Track Overview
Mohammad Aliannejadi, Zahra Abbasiantaeb, Shubham Chatterjee +2
Conversational Information Seeking has evolved rapidly in the last few years with the development of Large Language Models providing the basis for interpreting and responding in a…
Asking Multimodal Clarifying Questions in Mixed-Initiative Conversational Search
Yifei Yuan, Clemencia Siro, Mohammad Aliannejadi +2
In mixed-initiative conversational search systems, clarifying questions are used to help users who struggle to express their intentions in a single query. These questions aim to un…
Self-seeding and Multi-intent Self-instructing LLMs for Generating Intent-aware Information-Seeking dialogs
Arian Askari, Roxana Petcu, Chuan Meng +4
Identifying user intents in information-seeking dialogs is crucial for a system to meet user's information needs. Intent prediction (IP) is challenging and demands sufficient dialo…