68 citations · 177 across the 24 of their papers we have counts for
6 papers · 1 filter
Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
Pat Verga, Sebastian Hofstatter, Sophia Althammer +6
As Large Language Models (LLMs) have become more advanced, they have outpaced our abilities to accurately evaluate their quality. Not only is finding data to adequately probe parti…
Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Models
Xinyu Zhang, Sebastian Hofstätter, Patrick Lewis +2
Listwise rerankers based on large language models (LLM) are the zero-shot state-of-the-art. However, current works in this direction all depend on the GPT models, making it a singl…
Ranger: A Toolkit for Effect-Size Based Multi-Task Evaluation
Mete Sertkan, Sophia Althammer, Sebastian Hofstätter
In this paper, we introduce Ranger - a toolkit to facilitate the easy use of effect-size-based meta-analysis for multi-task evaluation in NLP and IR. We observed that our communiti…
FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation
Sebastian Hofstätter, Jiecao Chen, Karthik Raman +1
Retrieval-augmented generation models offer many benefits over standalone language models: besides a textual answer to a given query they provide provenance items retrieved from an…
Multi-Task Retrieval-Augmented Text Generation with Relevance Sampling
Sebastian Hofstätter, Jiecao Chen, Karthik Raman +1
This paper studies multi-task training of retrieval-augmented generation models for knowledge-intensive tasks. We propose to clean the training set by utilizing a distinct property…
Linguistically Informed Masking for Representation Learning in the Patent Domain
Sophia Althammer, Mark Buckley, Sebastian Hofstätter +1
Domain-specific contextualized language models have demonstrated substantial effectiveness gains for domain-specific downstream tasks, like similarity matching, entity recognition…