6 papers
Tracing Relational Knowledge Recall in Large Language Models
Nicholas PopoviÄ, Michael Färber
We study how large language models recall relational knowledge during text generation, with a focus on identifying latent representations suitable for relation classification via l…
Benchmarking Uncertainty Calibration in Large Language Model Long-Form Question Answering
Philip Müller, Nicholas PopoviÄ, Michael Färber +1
Large Language Models (LLMs) are commonly used in Question Answering (QA) settings, increasingly in the natural sciences if not science at large. Reliable Uncertainty Quantificatio…
Extractive Fact Decomposition for Interpretable Natural Language Inference in one Forward Pass
Nicholas PopoviÄ, Michael Färber
Recent works in Natural Language Inference (NLI) and related tasks, such as automated fact-checking, employ atomic fact decomposition to enhance interpretability and robustness. Fo…
Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation
Israfel Salazar, Manuel Fernández Burda, Shayekh Bin Islam +42
The evaluation of vision-language models (VLMs) has mainly relied on English-language benchmarks, leaving significant gaps in both multilingual and multicultural coverage. While mu…
Embedded Named Entity Recognition using Probing Classifiers
Nicholas PopoviÄ, Michael Färber
Streaming text generation has become a common way of increasing the responsiveness of language model powered applications, such as chat assistants. At the same time, extracting sem…
The Effects of Hallucinations in Synthetic Training Data for Relation Extraction
Steven Rogulsky, Nicholas Popovic, Michael Färber
Relation extraction is crucial for constructing knowledge graphs, with large high-quality datasets serving as the foundation for training, fine-tuning, and evaluating models. Gener…