5 citations · 9 across the 19 of their papers we have counts for
25 papers
Adapting Language Balance in Code-Switching Speech
Enes Yavuz Ugan, Ngoc-Quan Pham, Alexander Waibel
Despite achieving impressive results on standard benchmarks, large foundational models still struggle against code-switching test cases. When data scarcity cannot be used as the us…
Bayesian Low-Rank Factorization for Robust Model Adaptation
Enes Yavuz Ugan, Ngoc-Quan Pham, Alexander Waibel
Large speech foundation models achieve strong performance across many domains, but they often require adaptation to handle local needs such as code-switching, where speakers mix la…
A Cocktail-Party Benchmark: Multi-Modal dataset and Comparative Evaluation Results
Thai-Binh Nguyen, Katerina Zmolikova, Pingchuan Ma +3
We introduce the task of Multi-Modal Context-Aware Recognition (MCoRec) in the ninth CHiME Challenge, which addresses the cocktail-party problem of overlapping conversations in a s…
Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement
Tuan-Nam Nguyen, Ngoc-Quan Pham, Seymanur Akti +1
We propose a first streaming accent conversion (AC) model that transforms non-native speech into a native-like accent while preserving speaker identity, prosody and improving pronu…
Weight Factorization and Centralization for Continual Learning in Speech Recognition
Enes Yavuz Ugan, Ngoc-Quan Pham, Alexander Waibel
Modern neural network based speech recognition models are required to continually absorb new data without re-training the whole system, especially in downstream applications using…
Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness for Fine-tuning Foundation Models
Ngoc-Quan Pham, Tuan Truong, Quyen Tran +3
We introduce Interactive Bayesian Distributional Robustness (IBDR), a novel Bayesian inference framework that allows modeling the interactions between particles, thereby enhancing…
Cocktail-Party Audio-Visual Speech Recognition
Thai-Binh Nguyen, Ngoc-Quan Pham, Alexander Waibel
Audio-Visual Speech Recognition (AVSR) offers a robust solution for speech recognition in challenging environments, such as cocktail-party scenarios, where relying solely on audio…
PIER: A Novel Metric for Evaluating What Matters in Code-Switching
Enes Yavuz Ugan, Ngoc-Quan Pham, Leonard Bärmann +1
Code-switching, the alternation of languages within a single discourse, presents a significant challenge for Automatic Speech Recognition. Despite the unique nature of the task, pe…
Improving Pronunciation and Accent Conversion through Knowledge Distillation And Synthetic Ground-Truth from Native TTS
Tuan Nam Nguyen, Seymanur Akti, Ngoc Quan Pham +1
Previous approaches on accent conversion (AC) mainly aimed at making non-native speech sound more native while maintaining the original content and speaker identity. However, non-n…
Accent conversion using discrete units with parallel data synthesized from controllable accented TTS
Tuan Nam Nguyen, Ngoc Quan Pham, Alexander Waibel
The goal of accent conversion (AC) is to convert speech accents while preserving content and speaker identity. Previous methods either required reference utterances during inferenc…
Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages
Carlos Mullov, Ngoc-Quan Pham, Alexander Waibel
Multilingual neural machine translation systems learn to map sentences of different languages into a common representation space. Intuitively, with a growing number of seen languag…
Blending LLMs into Cascaded Speech Translation: KIT's Offline Speech Translation System for IWSLT 2024
Sai Koneru, Thai-Binh Nguyen, Ngoc-Quan Pham +4
Large Language Models (LLMs) are currently under exploration for various tasks, including Automatic Speech Recognition (ASR), Machine Translation (MT), and even End-to-End Speech T…
End-to-End Evaluation for Low-Latency Simultaneous Speech Translation
Christian Huber, Tu Anh Dinh, Carlos Mullov +10
The challenge of low-latency speech translation has recently draw significant interest in the research community as shown by several publications and shared tasks. Therefore, it is…
KIT's Multilingual Speech Translation System for IWSLT 2023
Danni Liu, Thai Binh Nguyen, Sai Koneru +7
Many existing speech translation benchmarks focus on native-English speech in high-quality recording conditions, which often do not match the conditions in real-life use-cases. In…
Adaptive multilingual speech recognition with pretrained models
Ngoc-Quan Pham, Alex Waibel, Jan Niehues
Multilingual speech recognition with supervised learning has achieved great results as reflected in recent research. With the development of pretraining methods on audio and text d…
Efficient Weight factorization for Multilingual Speech Recognition
Ngoc-Quan Pham, Tuan-Nam Nguyen, Sebastian Stueker +1
End-to-end multilingual speech recognition involves using a single model training on a compositional speech corpus including many languages, resulting in a single neural network to…
Unsupervised Transfer Learning in Multilingual Neural Machine Translation with Cross-Lingual Word Embeddings
Carlos Mullov, Ngoc-Quan Pham, Alexander Waibel
In this work we look into adding a new language to a multilingual NMT system in an unsupervised fashion. Under the utilization of pre-trained cross-lingual word embeddings we seek…
Relative Positional Encoding for Speech Recognition and Direct Translation
Ngoc-Quan Pham, Thanh-Le Ha, Tuan-Nam Nguyen +5
Transformer models are powerful sequence-to-sequence architectures that are capable of directly mapping speech inputs to transcriptions or translations. However, the mechanism for…
High Performance Sequence-to-Sequence Model for Streaming Speech Recognition
Thai-Son Nguyen, Ngoc-Quan Pham, Sebastian Stueker +1
Recently sequence-to-sequence models have started to achieve state-of-the-art performance on standard speech recognition tasks when processing audio data in batch mode, i.e., the c…
Modeling Confidence in Sequence-to-Sequence Models
Jan Niehues, Ngoc-Quan Pham
Recently, significant improvements have been achieved in various natural language processing tasks using neural sequence-to-sequence models. While aiming for the best generation qu…
Improving Zero-shot Translation with Language-Independent Constraints
Ngoc-Quan Pham, Jan Niehues, Thanh-Le Ha +1
An important concern in training multilingual neural machine translation (NMT) is to translate between language pairs unseen during training, i.e zero-shot translation. Improving t…
Self-Attentional Models for Lattice Inputs
Matthias Sperber, Graham Neubig, Ngoc-Quan Pham +1
Lattices are an efficient and effective method to encode ambiguity of upstream systems in natural language processing tasks, for example to compactly capture multiple speech recogn…
Very Deep Self-Attention Networks for End-to-End Speech Recognition
Ngoc-Quan Pham, Thai-Son Nguyen, Jan Niehues +3
Recently, end-to-end sequence-to-sequence models for speech recognition have gained significant interest in the research community. While previous architecture choices revolve arou…
Towards one-shot learning for rare-word translation with external experts
Ngoc-Quan Pham, Jan Niehues, Alex Waibel
Neural machine translation (NMT) has significantly improved the quality of automatic translation models. One of the main challenges in current systems is the translation of rare wo…