6 papers · 1 filter
Does task decomposition improve automatic NLG evaluation?
Sebastian Steindl, Nikos Voskarides, Alberto Gasparin +1
The LLM-as-a-judge (LLMaJ) framework has emerged as a promising solution for cheap, reproducible, reference-free Natural Language Generation (NLG) evaluation. Prior work seeks to i…
Post-hoc Alignment of LLM-judges to Human Judgment Distribution
Sebastian Steindl, Nikos Voskarides, Alberto Gasparin +1
The LLM-as-a-judge (LLMaJ) framework offers a cost-effective and reproducible solution for automatic evaluation. However, current evaluation practices typically compare LLMaJ judgm…
Multimodal LLMs are not all you need for Pediatric Speech Language Pathology
Darren Fürst, Sebastian Steindl, Ulrich Schäfer
Speech Sound Disorders (SSD) affect roughly five percent of children, yet speech-language pathologists face severe staffing shortages and unmanageable caseloads. We test a hierarch…
MonoTODia: Translating Monologue Requests to Task-Oriented Dialogues
Sebastian Steindl, Ulrich Schäfer, Bernd Ludwig
Data scarcity is one of the main problems when it comes to real-world applications of transformer-based models. This is especially evident for task-oriented dialogue (TOD) systems,…
Question: How do Large Language Models perform on the Question Answering tasks? Answer:
Kevin Fischer, Darren Fürst, Sebastian Steindl +2
Large Language Models (LLMs) have been showing promising results for various NLP-tasks without the explicit need to be trained for these tasks by using few-shot or zero-shot prompt…
CoPrUS: Consistency Preserving Utterance Synthesis towards more realistic benchmark dialogues
Sebastian Steindl, Ulrich Schäfer, Bernd Ludwig
Large-scale Wizard-Of-Oz dialogue datasets have enabled the training of deep learning-based dialogue systems. While they are successful as benchmark datasets, they lack certain typ…