7 papers
Review Arcade: On the Human Alignment and Gameability of LLM Reviews
Hans Ole Hatzel, Sebastian Steindl, Jan Strich
LLM-generated reviews for scientific papers are gaining considerable traction and are even being officially piloted by major conferences. We have to assume that not only reviewers…
Self-Calibrating Language Models via Test-Time Discriminative Distillation
Mohamed Rissal Hedna, Jan Strich, Martin Semmann +1
Large language models (LLMs) are systematically overconfident: they routinely express high certainty on questions they often answer incorrectly. Existing calibration methods either…
LEMUR: A Corpus for Robust Fine-Tuning of Multilingual Law Embedding Models for Retrieval
Narges Baba Ahmadi, Jan Strich, Martin Semmann +1
Large language models (LLMs) are increasingly used to access legal information. Yet, their deployment in multilingual legal settings is constrained by unreliable retrieval and the…
Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QA
Klejda Alushi, Jan Strich, Chris Biemann +1
Conversational question answering increasingly relies on retrieval-augmented generation (RAG) to ground large language models (LLMs) in external knowledge. Yet, most existing studi…
T-RAGBench: Text-and-Table Benchmark for Evaluating Retrieval-Augmented Generation
Jan Strich, Enes Kutay Isgorur, Maximilian Trescher +2
Since many real-world documents combine textual and tabular data, robust Retrieval Augmented Generation (RAG) systems are essential for effectively accessing and analyzing such con…
EncouRAGe: Evaluating RAG Local, Fast, and Reliable
Jan Strich, Adeline Scharfenberg, Chris Biemann +1
We introduce EncouRAGe, a comprehensive Python framework designed to streamline the development and evaluation of Retrieval-Augmented Generation (RAG) systems using Large Language…