9 citations · 10 across the 18 of their papers we have counts for
21 papers
Loud and Clear: Dynamic Activation Steering for Improving Speech Intelligibility in Noisy Environments
Seymanur Akti, Alexander Waibel
Speech becomes less intelligible in noisy environments, and humans naturally adapt their voice to compensate. Inspired by this behavior, we investigate whether a text-to-speech (TT…
From Speech to Interaction: Analyzing Multimodal Systems in Cocktail-Party Scenarios
Thai-Binh Nguyen, Zhaolin Li, Jan Niehues +1
Humans have the remarkable ability to engage in spontaneous informal conversations and selectively attend to individual speakers while filtering out competing speech from nearby co…
Synthesizing the Lombard Effect: Multi-Level Control of Speech Clarity and Vocal Effort in TTS
Seymanur Akti, Alexander Waibel
Humans tend to speak louder and clearer in challenging environments, such as noisy conditions or when addressing hearingimpaired listeners, which is called Lombard effect. To simul…
Adding Robust Code-Switching Capabilities to High Performance Multilingual ASR
Enes Yavuz Ugan, Alexander Waibel
Code-switching (CSW) remains challenging for large multi-lingual ASR systems in real-world deployment. While fine-tuning on synthetic CSW data is possible, it generally degrades st…
KIT's Submission to Cross-Lingual Voice Cloning in IWSLT 2026
Seymanur Akti, Alexander Waibel
Cross-lingual voice cloning aims to generate speech in a target language while preserving speaker identity from a source-language reference. This task is central to speech translat…
Zero-Shot Lombard Speech Synthesis with Controllable Style Embeddings
Seymanur Akti, Alexander Waibel
The Lombard effect plays a key role in natural communication, particularly in noisy environments or when addressing hearing-impaired listeners. We present a controllable text-to-sp…
Adapting Language Balance in Code-Switching Speech
Enes Yavuz Ugan, Ngoc-Quan Pham, Alexander Waibel
Despite achieving impressive results on standard benchmarks, large foundational models still struggle against code-switching test cases. When data scarcity cannot be used as the us…
Bayesian Low-Rank Factorization for Robust Model Adaptation
Enes Yavuz Ugan, Ngoc-Quan Pham, Alexander Waibel
Large speech foundation models achieve strong performance across many domains, but they often require adaptation to handle local needs such as code-switching, where speakers mix la…
A Cocktail-Party Benchmark: Multi-Modal dataset and Comparative Evaluation Results
Thai-Binh Nguyen, Katerina Zmolikova, Pingchuan Ma +3
We introduce the task of Multi-Modal Context-Aware Recognition (MCoRec) in the ninth CHiME Challenge, which addresses the cocktail-party problem of overlapping conversations in a s…
Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement
Tuan-Nam Nguyen, Ngoc-Quan Pham, Seymanur Akti +1
We propose a first streaming accent conversion (AC) model that transforms non-native speech into a native-like accent while preserving speaker identity, prosody and improving pronu…
Weight Factorization and Centralization for Continual Learning in Speech Recognition
Enes Yavuz Ugan, Ngoc-Quan Pham, Alexander Waibel
Modern neural network based speech recognition models are required to continually absorb new data without re-training the whole system, especially in downstream applications using…
Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversion
Seymanur Akti, Tuan Nam Nguyen, Alexander Waibel
Expressive voice conversion aims to transfer both speaker identity and expressive attributes from a target speech to a given source speech. In this work, we improve over a self-sup…
Cocktail-Party Audio-Visual Speech Recognition
Thai-Binh Nguyen, Ngoc-Quan Pham, Alexander Waibel
Audio-Visual Speech Recognition (AVSR) offers a robust solution for speech recognition in challenging environments, such as cocktail-party scenarios, where relying solely on audio…
KIT's Low-resource Speech Translation Systems for IWSLT2025: System Enhancement with Synthetic Data and Model Regularization
Zhaolin Li, Yining Liu, Danni Liu +6
This paper presents KIT's submissions to the IWSLT 2025 low-resource track. We develop both cascaded systems, consisting of Automatic Speech Recognition (ASR) and Machine Translati…
MSA-ASR: Efficient Multilingual Speaker Attribution with frozen ASR Models
Thai-Binh Nguyen, Alexander Waibel
Speaker-attributed automatic speech recognition (SA-ASR) aims to transcribe speech while assigning transcripts to the corresponding speakers accurately. Existing methods often rely…
Improving Pronunciation and Accent Conversion through Knowledge Distillation And Synthetic Ground-Truth from Native TTS
Tuan Nam Nguyen, Seymanur Akti, Ngoc Quan Pham +1
Previous approaches on accent conversion (AC) mainly aimed at making non-native speech sound more native while maintaining the original content and speaker identity. However, non-n…
Accent conversion using discrete units with parallel data synthesized from controllable accented TTS
Tuan Nam Nguyen, Ngoc Quan Pham, Alexander Waibel
The goal of accent conversion (AC) is to convert speech accents while preserving content and speaker identity. Previous methods either required reference utterances during inferenc…
Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages
Carlos Mullov, Ngoc-Quan Pham, Alexander Waibel
Multilingual neural machine translation systems learn to map sentences of different languages into a common representation space. Intuitively, with a growing number of seen languag…
Blending LLMs into Cascaded Speech Translation: KIT's Offline Speech Translation System for IWSLT 2024
Sai Koneru, Thai-Binh Nguyen, Ngoc-Quan Pham +4
Large Language Models (LLMs) are currently under exploration for various tasks, including Automatic Speech Recognition (ASR), Machine Translation (MT), and even End-to-End Speech T…
Super-Human Performance in Online Low-latency Recognition of Conversational Speech
Thai-Son Nguyen, Sebastian Stueker, Alex Waibel
Achieving super-human performance in recognizing human speech has been a goal for several decades, as researchers have worked on increasingly challenging tasks. In the 1990's it wa…
Bimodal Speech Emotion Recognition Using Pre-Trained Language Models
Verena Heusser, Niklas Freymuth, Stefan Constantin +1
Speech emotion recognition is a challenging task and an important step towards more natural human-machine interaction. We show that pre-trained language models can be fine-tuned fo…