68 citations · 177 across the 24 of their papers we have counts for
30 papers
Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
Pat Verga, Sebastian Hofstatter, Sophia Althammer +6
As Large Language Models (LLMs) have become more advanced, they have outpaced our abilities to accurately evaluate their quality. Not only is finding data to adequately probe parti…
Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Models
Xinyu Zhang, Sebastian Hofstätter, Patrick Lewis +2
Listwise rerankers based on large language models (LLM) are the zero-shot state-of-the-art. However, current works in this direction all depend on the GPT models, making it a singl…
Annotating Data for Fine-Tuning a Neural Ranker? Current Active Learning Strategies are not Better than Random Selection
Sophia Althammer, Guido Zuccon, Sebastian Hofstätter +2
Search methods based on Pretrained Language Models (PLM) have demonstrated great effectiveness gains compared to statistical and early neural ranking models. However, fine-tuning P…
Ranger: A Toolkit for Effect-Size Based Multi-Task Evaluation
Mete Sertkan, Sophia Althammer, Sebastian Hofstätter
In this paper, we introduce Ranger - a toolkit to facilitate the easy use of effect-size-based meta-analysis for multi-task evaluation in NLP and IR. We observed that our communiti…
FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation
Sebastian Hofstätter, Jiecao Chen, Karthik Raman +1
Retrieval-augmented generation models offer many benefits over standalone language models: besides a textual answer to a given query they provide provenance items retrieved from an…
TripJudge: A Relevance Judgement Test Collection for TripClick Health Retrieval
Sophia Althammer, Sebastian Hofstätter, Suzan Verberne +1
Robust test collections are crucial for Information Retrieval research. Recently there is a growing interest in evaluating retrieval systems for domain-specific retrieval tasks, ho…
Multi-Task Retrieval-Augmented Text Generation with Relevance Sampling
Sebastian Hofstätter, Jiecao Chen, Karthik Raman +1
This paper studies multi-task training of retrieval-augmented generation models for knowledge-intensive tasks. We propose to clean the training set by utilizing a distinct property…
Are We There Yet? A Decision Framework for Replacing Term Based Retrieval with Dense Retrieval Systems
Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra +2
Recently, several dense retrieval (DR) models have demonstrated competitive performance to term-based retrieval that are ubiquitous in search systems. In contrast to term-based mat…
Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction
Sebastian Hofstätter, Omar Khattab, Sophia Althammer +2
Recent progress in neural information retrieval has demonstrated large gains in effectiveness, while often sacrificing the efficiency and interpretability of the neural model compa…
Establishing Strong Baselines for TripClick Health Retrieval
Sebastian Hofstätter, Sophia Althammer, Mete Sertkan +1
We present strong Transformer-based re-ranking and dense retrieval baselines for the recently released TripClick health ad-hoc retrieval collection. We improve the - originally too…
PARM: A Paragraph Aggregation Retrieval Model for Dense Document-to-Document Retrieval
Sophia Althammer, Sebastian Hofstätter, Mete Sertkan +2
Dense passage retrieval (DPR) models show great effectiveness gains in first stage retrieval for the web domain. However in the web domain we are in a setting with large amounts of…
A Time-Optimized Content Creation Workflow for Remote Teaching
Sebastian Hofstätter, Sophia Althammer, Mete Sertkan +1
We describe our workflow to create an engaging remote learning experience for a university course, while minimizing the post-production time of the educators. We make use of ubiqui…
Linguistically Informed Masking for Representation Learning in the Patent Domain
Sophia Althammer, Mark Buckley, Sebastian Hofstätter +1
Domain-specific contextualized language models have demonstrated substantial effectiveness gains for domain-specific downstream tasks, like similarity matching, entity recognition…
Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling
Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang +2
A vital step towards the widespread adoption of neural retrieval models is their resource efficiency throughout the training, indexing and query workflows. The neural IR community…
Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
Sebastian Hofstätter, Bhaskar Mitra, Hamed Zamani +2
An emerging recipe for achieving state-of-the-art effectiveness in neural document re-ranking involves utilizing large pre-trained language models - e.g., BERT - to evaluate all in…
Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani +1
The Transformer-Kernel (TK) model has demonstrated strong reranking performance on the TREC Deep Learning benchmark -- and can be considered to be an efficient (but slightly less e…
Cross-domain Retrieval in the Legal and Patent Domains: a Reproducibility Study
Sophia Althammer, Sebastian Hofstätter, Allan Hanbury
Domain specific search has always been a challenging information retrieval task due to several challenges such as the domain specific language, the unique task setting, as well as…
Mitigating the Position Bias of Transformer Models in Passage Re-Ranking
Sebastian Hofstätter, Aldo Lipani, Sophia Althammer +2
Supervised machine learning models and their evaluation strongly depends on the quality of the underlying dataset. When we search for a relevant piece of information it may appear…
Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani +1
We benchmark Conformer-Kernel models under the strict blind evaluation setting of the TREC 2020 Deep Learning track. In particular, we study the impact of incorporating: (i) Explic…
Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation
Sebastian Hofstätter, Sophia Althammer, Michael Schröder +2
Retrieval and ranking models are the backbone of many applications such as web search, open domain QA, or text-based recommender systems. The latency of neural ranking models at qu…
Fine-Grained Relevance Annotations for Multi-Task Document Ranking and Question Answering
Sebastian Hofstätter, Markus Zlabinger, Mete Sertkan +2
There are many existing retrieval and question answering datasets. However, most of them either focus on ranked list evaluation or single-candidate question answering. This divide…
Conformer-Kernel with Query Term Independence for Document Retrieval
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani +1
The Transformer-Kernel (TK) model has demonstrated strong reranking performance on the TREC Deep Learning benchmark---and can be considered to be an efficient (but slightly less ef…
DEXA: Supporting Non-Expert Annotators with Dynamic Examples from Experts
Markus Zlabinger, Marta Sabou, Sebastian Hofstätter +2
The success of crowdsourcing based annotation of text corpora depends on ensuring that crowdworkers are sufficiently well-trained to perform the annotation task accurately. To that…
Local Self-Attention over Long Text for Efficient Document Retrieval
Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra +2
Neural networks, particularly Transformer-based architectures, have achieved significant performance improvements on several retrieval benchmarks. When the items being retrieved ar…