8 citations · 28 across the 7 of their papers we have counts for
7 papers
When Attention Goes Blind: Numerical Failure in ALiBi Positional Encodings
Christopher Schröder, Lukas Gienapp, Ferdinand Schlatt +2
We identify a previously overlooked failure mode of ALiBi positional encoding: its linear bias scaling underflows floating-point precision, which zeroes out a large fraction of att…
The German Commons - 154 Billion Tokens of Openly Licensed Text for German Language Models
Lukas Gienapp, Christopher Schröder, Stefan Schweter +5
Large language model development relies on large-scale training corpora, yet most contain data of unclear licensing status, limiting the development of truly open models. This prob…
Variations in Relevance Judgments and the Shelf Life of Test Collections
Andrew Parry, Maik Fröbe, Harrisen Scells +5
The fundamental property of Cranfield-style evaluations, that system rankings are stable even when assessors disagree on individual relevance decisions, was validated on traditiona…
Lightning IR: Straightforward Fine-tuning and Inference of Transformer-based Language Models for Information Retrieval
Ferdinand Schlatt, Maik Fröbe, Matthias Hagen
A wide range of transformer-based language models have been proposed for information retrieval tasks. However, including transformer-based models in retrieval pipelines is often co…
Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-Ranking
Ferdinand Schlatt, Maik Fröbe, Harrisen Scells +6
Cross-encoders distilled from large language models (LLMs) are often more effective re-rankers than cross-encoders fine-tuned on manually labeled data. However, distilled models do…
Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders
Ferdinand Schlatt, Maik Fröbe, Harrisen Scells +6
Existing cross-encoder models can be categorized as pointwise, pairwise, or listwise. Pairwise and listwise models allow passage interactions, which typically makes them more effec…
Investigating the Effects of Sparse Attention on Cross-Encoders
Ferdinand Schlatt, Maik Fröbe, Matthias Hagen
Cross-encoders are effective passage and document re-rankers but less efficient than other neural or classic retrieval models. A few previous studies have applied windowed self-att…