6 papers
OmniFusion: Simultaneous Multilingual Multimodal Translations via Modular Fusion
Sai Koneru, Matthias Huck, Jan Niehues
There has been significant progress in open-source text-only translation large language models (LLMs) with better language coverage and quality. However, these models can be only u…
BOOM: Beyond Only One Modality KIT's Multimodal Multilingual Lecture Companion
Sai Koneru, Fabian Retkowski, Christian Huber +5
The globalization of education and rapid growth of online learning have made localizing educational content a critical challenge. Lecture materials are inherently multimodal, combi…
End-to-End Evaluation for Low-Latency Simultaneous Speech Translation
Christian Huber, Tu Anh Dinh, Carlos Mullov +10
The challenge of low-latency speech translation has recently draw significant interest in the research community as shown by several publications and shared tasks. Therefore, it is…
Quality-Aware Decoding: Unifying Quality Estimation and Decoding
Sai Koneru, Matthias Huck, Miriam Exel +1
Quality Estimation (QE) models for Neural Machine Translation (NMT) predict the quality of the hypothesis without having access to the reference. An emerging research direction in…
KIT's Offline Speech Translation and Instruction Following Submission for IWSLT 2025
Sai Koneru, Maike Züfle, Thai-Binh Nguyen +3
The scope of the International Workshop on Spoken Language Translation (IWSLT) has recently broadened beyond traditional Speech Translation (ST) to encompass a wider array of tasks…
Plug, Play, and Fuse: Zero-Shot Joint Decoding via Word-Level Re-ranking Across Diverse Vocabularies
Sai Koneru, Matthias Huck, Miriam Exel +1
Recent advancements in NLP have resulted in models with specialized strengths, such as processing multimodal inputs or excelling in specific domains. However, real-world tasks, lik…