9 citations · 43 across the 9 of their papers we have counts for
9 papers
The Viability of Crowdsourcing for RAG Evaluation
Lukas Gienapp, Tim Hagen, Maik Fröbe +4
How good are humans at writing and judging responses in retrieval-augmented generation (RAG) scenarios? To answer this question, we investigate the efficacy of crowdsourcing for RA…
Counterfactual Query Rewriting to Use Historical Relevance Feedback
Jüri Keller, Maik Fröbe, Gijs Hendriksen +4
When a retrieval system receives a query it has encountered before, previous relevance feedback, such as clicks or explicit judgments can help to improve retrieval results. However…
Lightning IR: Straightforward Fine-tuning and Inference of Transformer-based Language Models for Information Retrieval
Ferdinand Schlatt, Maik Fröbe, Matthias Hagen
A wide range of transformer-based language models have been proposed for information retrieval tasks. However, including transformer-based models in retrieval pipelines is often co…
Systematic Evaluation of Neural Retrieval Models on the Touché 2020 Argument Retrieval Subset of BEIR
Nandan Thakur, Luiz Bonifacio, Maik Fröbe +5
The zero-shot effectiveness of neural retrieval models is often evaluated on the BEIR benchmark -- a combination of different IR evaluation datasets. Interestingly, previous studie…
Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance
Kaitlyn Zhou, Jena D. Hwang, Xiang Ren +3
The ability to communicate uncertainty, risk, and limitation is crucial for the safety of large language models. However, current evaluations of these abilities rely on simple cali…
Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-Ranking
Ferdinand Schlatt, Maik Fröbe, Harrisen Scells +6
Cross-encoders distilled from large language models (LLMs) are often more effective re-rankers than cross-encoders fine-tuned on manually labeled data. However, distilled models do…
Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders
Ferdinand Schlatt, Maik Fröbe, Harrisen Scells +6
Existing cross-encoder models can be categorized as pointwise, pairwise, or listwise. Pairwise and listwise models allow passage interactions, which typically makes them more effec…
Analyzing Adversarial Attacks on Sequence-to-Sequence Relevance Models
Andrew Parry, Maik Fröbe, Sean MacAvaney +2
Modern sequence-to-sequence relevance models like monoT5 can effectively capture complex textual interactions between queries and documents through cross-encoding. However, the use…
Investigating the Effects of Sparse Attention on Cross-Encoders
Ferdinand Schlatt, Maik Fröbe, Matthias Hagen
Cross-encoders are effective passage and document re-rankers but less efficient than other neural or classic retrieval models. A few previous studies have applied windowed self-att…