29 citations · 36 across the 15 of their papers we have counts for
16 papers
Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following
Niklas Scholz, David Thulke, Abdallah Nasir +3
Fine-tuning large language models on parallel data improves translation quality but can cause catastrophic forgetting. Mitigation methods are generally evaluated by retention on ge…
Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Language
Kristin Gnadt, David Thulke, Simone Kopeinik +1
In recent years, various methods have been proposed to evaluate gender bias in large language models (LLMs). A key challenge lies in the transferability of bias measurement methods…
Listen to the Context: Towards Faithful Large Language Models for Retrieval Augmented Generation on Climate Questions
David Thulke, Jakob Kemmler, Christian Dugast +1
Large language models that use retrieval augmented generation have the potential to unlock valuable knowledge for researchers, policymakers, and the public by making long and techn…
Prompting and Fine-Tuning of Small LLMs for Length-Controllable Telephone Call Summarization
David Thulke, Yingbo Gao, Rricha Jalota +2
This paper explores the rapid development of a telephone call summarization system utilizing large language models (LLMs). Our approach involves initial experiments with prompting…
Medical Spoken Named Entity Recognition
Khai Le-Duc, David Thulke, Hung-Phong Tran +4
Spoken Named Entity Recognition (NER) aims to extract named entities from speech and categorise them into types like person, location, organization, etc. In this work, we present V…
ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Change
David Thulke, Yingbo Gao, Petrus Pelser +23
This paper introduces ClimateGPT, a model family of domain-specific large language models that synthesize interdisciplinary research on climate change. We trained two 7B models fro…
Leveraging Cross-Lingual Transfer Learning in Spoken Named Entity Recognition Systems
Moncef Benaicha, David Thulke, M. A. Tuğtekin Turan
Recent Named Entity Recognition (NER) advancements have significantly enhanced text classification capabilities. This paper focuses on spoken NER, aimed explicitly at spoken docume…
Task-oriented Document-Grounded Dialog Systems by HLTPR@RWTH for DSTC9 and DSTC10
David Thulke, Nico Daheim, Christian Dugast +1
This paper summarizes our contributions to the document-grounded dialog tasks at the 9th and 10th Dialog System Technology Challenges (DSTC9 and DSTC10). In both iterations the tas…
Does Joint Training Really Help Cascaded Speech Translation?
Viet Anh Khoa Tran, David Thulke, Yingbo Gao +2
Currently, in speech translation, the straightforward approach - cascading a recognition system with a translation system - delivers state-of-the-art results. However, fundamental…
Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token
Baohao Liao, David Thulke, Sanjika Hewavitharana +2
The pre-training of masked language models (MLMs) consumes massive computation to achieve good results on downstream NLP tasks, resulting in a large carbon footprint. In the vanill…
Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model
Nico Daheim, David Thulke, Christian Dugast +1
In this work, we present a model for document-grounded response generation in dialog that is decomposed into two components according to Bayes theorem. One component is a tradition…
Adapting Document-Grounded Dialog Systems to Spoken Conversations using Data Augmentation and a Noisy Channel Model
David Thulke, Nico Daheim, Christian Dugast +1
This paper summarizes our submission to Task 2 of the second track of the 10th Dialog System Technology Challenge (DSTC10) "Knowledge-grounded Task-oriented Dialogue Modeling on Sp…
Investigation on Data Adaptation Techniques for Neural Named Entity Recognition
Evgeniia Tokarchuk, David Thulke, Weiyue Wang +2
Data processing is an important step in various natural language processing tasks. As the commonly used datasets in named entity recognition contain only a limited number of sample…
Cascaded Span Extraction and Response Generation for Document-Grounded Dialog
Nico Daheim, David Thulke, Christian Dugast +1
This paper summarizes our entries to both subtasks of the first DialDoc shared task which focuses on the agent response prediction task in goal-oriented document-grounded dialogs.…
On Sampling-Based Training Criteria for Neural Language Modeling
Yingbo Gao, David Thulke, Alexander Gerstenberger +3
As the vocabulary size of modern word-based language models becomes ever larger, many sampling-based training criteria are proposed and investigated. The essence of these sampling…
Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog
David Thulke, Nico Daheim, Christian Dugast +1
This paper summarizes our work on the first track of the ninth Dialog System Technology Challenge (DSTC 9), "Beyond Domain APIs: Task-oriented Conversational Modeling with Unstruct…