11 citations · 11 across the 9 of their papers we have counts for
15 papers
Calibrating LLM Judges for Human and AI Conversations
Maike Züfle, Patrícia Schmidtová, Vilém Zouhar +9
Measuring how successful a conversation is remains difficult, even for humans judging spoken dialogue. We evaluate state-of-the-art LLMs as pointwise and pairwise judges of convers…
Controlling Backchannels in Streamable Full-duplex Models
Maike Züfle, Peter Polák, Sefik Emre Eskimez +3
Backchannels, brief acknowledgements like "uh-huh" produced while the other party may still be talking, are central to natural conversation, but full-duplex spoken dialogue models…
Distributional Metrics for Evaluating Spoken Conversational Systems
Shree Harsha Bokkahalli Satish, Erica Cooper, Patrícia Schmidtová +4
Evaluating conversational systems is a difficult and unresolved problem. We introduce the Conversational Distribution Score (CDS), which compares distributions of conversational be…
Why We Need Speech to Evaluate Speech Translation
Maike Züfle, Danni Liu, Vilém Zouhar +1
Speech translation models are increasingly capable of preserving speech-specific information (e.g., speaker gender, prosody, and emphasis), yet evaluation metrics remain blind to s…
Do What I Say: A Spoken Prompt Dataset for Instruction-Following
Maike Züfle, Sara Papi, Fabian Retkowski +5
Speech Large Language Models (SLLMs) have rapidly expanded, supporting a wide range of tasks. These models are typically evaluated using text prompts, which may not reflect real-wo…
F-Actor: Controllable Conversational Behaviour in Full-Duplex Models
Maike Züfle, Ondrej Klejch, Nicholas Sanders +3
Spoken conversational systems require more than accurate speech generation to have human-like conversations: to feel natural and engaging, they must produce conversational behaviou…
Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs
Sara Papi, Javier Garcia Gilabert, Zachary Hopton +8
As Large Language Models (LLMs) expand beyond text, integrating speech as a native modality has given rise to SpeechLLMs, which directly process spoken language and enable speech-t…
COMET-poly: Machine Translation Metric Grounded in Other Candidates
Maike Züfle, Vilém Zouhar, Tu Anh Dinh +3
Automated metrics for machine translation attempt to replicate human judgment. Unlike humans, who often assess a translation in the context of multiple alternatives, these metrics…
MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks
Sara Papi, Maike Züfle, Marco Gaido +5
Recent advances in large language models have laid the foundation for multimodal LLMs (MLLMs), which unify text, speech, and vision within a single framework. As these models are r…
KIT's Offline Speech Translation and Instruction Following Submission for IWSLT 2025
Sai Koneru, Maike Züfle, Thai-Binh Nguyen +3
The scope of the International Workshop on Spoken Language Translation (IWSLT) has recently broadened beyond traditional Speech Translation (ST) to encompass a wider array of tasks…
NUTSHELL: A Dataset for Abstract Generation from Scientific Talks
Maike Züfle, Sara Papi, Beatrice Savoldi +3
Scientific communication is receiving increasing attention in natural language processing, especially to help researches access, summarize, and generate content. One emerging appli…
Early-Exit and Instant Confidence Translation Quality Estimation
Vilém Zouhar, Maike Züfle, Beni Egressy +3
Quality estimation is omnipresent in machine translation, for both evaluation and generation. Unfortunately, quality estimation models are often opaque and computationally expensiv…
A Bayesian Optimization Approach to Machine Translation Reranking
Julius Cheng, Maike Züfle, Vilém Zouhar +1
Reranking a list of candidates from a machine translation system with an external scoring model and returning the highest-scoring candidate remains a simple and effective method fo…
Latent Feature-based Data Splits to Improve Generalisation Evaluation: A Hate Speech Detection Case Study
Maike Züfle, Verna Dankers, Ivan Titov
With the ever-growing presence of social media platforms comes the increased spread of harmful content and the need for robust hate speech detection systems. Such systems easily ov…
Text-to-OverpassQL: A Natural Language Interface for Complex Geodata Querying of OpenStreetMap
Michael Staniek, Raphael Schumann, Maike Züfle +1
We present Text-to-OverpassQL, a task designed to facilitate a natural language interface for querying geodata from OpenStreetMap (OSM). The Overpass Query Language (OverpassQL) al…