4 papers
Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following
Niklas Scholz, David Thulke, Abdallah Nasir +3
Fine-tuning large language models on parallel data improves translation quality but can cause catastrophic forgetting. Mitigation methods are generally evaluated by retention on ge…
Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning
Arthur Becker, Jakob Kemmler, David Thulke +3
Supervised fine-tuning (SFT) trains a base language model to imitate target responses, and these targets may require knowledge the base model has not robustly internalized. We stud…
Listen to the Context: Towards Faithful Large Language Models for Retrieval Augmented Generation on Climate Questions
David Thulke, Jakob Kemmler, Christian Dugast +1
Large language models that use retrieval augmented generation have the potential to unlock valuable knowledge for researchers, policymakers, and the public by making long and techn…
Prompting and Fine-Tuning of Small LLMs for Length-Controllable Telephone Call Summarization
David Thulke, Yingbo Gao, Rricha Jalota +2
This paper explores the rapid development of a telephone call summarization system utilizing large language models (LLMs). Our approach involves initial experiments with prompting…