10 citations · 51 across the 23 of their papers we have counts for
11 papers · 1 filter
Ranking Generated Answers: On the Agreement of Retrieval Models with Humans on Consumer Health Questions
Sebastian Heineking, Jonas Probst, Daniel Steinbach +2
Evaluating the output of generative large language models (LLMs) is challenging and difficult to scale. Many evaluations of LLMs focus on tasks such as single-choice question-answe…
Systematic Evaluation of Neural Retrieval Models on the Touché 2020 Argument Retrieval Subset of BEIR
Nandan Thakur, Luiz Bonifacio, Maik Fröbe +5
The zero-shot effectiveness of neural retrieval models is often evaluated on the BEIR benchmark -- a combination of different IR evaluation datasets. Interestingly, previous studie…
Learning Effective Representations for Retrieval Using Self-Distillation with Adaptive Relevance Margins
Lukas Gienapp, Niklas Deckers, Martin Potthast +1
Representation-based retrieval models, so-called bi-encoders, estimate the relevance of a document to a query by calculating the similarity of their respective embeddings. Current…
Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance
Kaitlyn Zhou, Jena D. Hwang, Xiang Ren +3
The ability to communicate uncertainty, risk, and limitation is crucial for the safety of large language models. However, current evaluations of these abilities rely on simple cali…
Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-Ranking
Ferdinand Schlatt, Maik Fröbe, Harrisen Scells +6
Cross-encoders distilled from large language models (LLMs) are often more effective re-rankers than cross-encoders fine-tuned on manually labeled data. However, distilled models do…
If there's a Trigger Warning, then where's the Trigger? Investigating Trigger Warnings at the Passage Level
Matti Wiegmann, Jennifer Rakete, Magdalena Wolska +2
Trigger warnings are labels that preface documents with sensitive content if this content could be perceived as harmful by certain groups of readers. Since warnings about a documen…
Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders
Ferdinand Schlatt, Maik Fröbe, Harrisen Scells +6
Existing cross-encoder models can be categorized as pointwise, pairwise, or listwise. Pairwise and listwise models allow passage interactions, which typically makes them more effec…
Task-Oriented Paraphrase Analytics
Marcel Gohsen, Matthias Hagen, Martin Potthast +1
Since paraphrasing is an ill-defined task, the term "paraphrasing" covers text transformation tasks with different characteristics. Consequently, existing paraphrasing studies have…
Analyzing Adversarial Attacks on Sequence-to-Sequence Relevance Models
Andrew Parry, Maik Fröbe, Sean MacAvaney +2
Modern sequence-to-sequence relevance models like monoT5 can effectively capture complex textual interactions between queries and documents through cross-encoding. However, the use…
Detecting Generated Native Ads in Conversational Search
Sebastian Schmidt, Ines Zelch, Janek Bevendorff +3
Conversational search engines such as YouChat and Microsoft Copilot use large language models (LLMs) to generate responses to queries. It is only a small step to also let the same…
Zero-shot Generative Large Language Models for Systematic Review Screening Automation
Shuai Wang, Harrisen Scells, Shengyao Zhuang +3
Systematic reviews are crucial for evidence-based medicine as they comprehensively analyse published research findings on specific questions. Conducting such reviews is often resou…