11 citations · 30 across the 15 of their papers we have counts for
16 papers
Teaching a Multilingual Large Language Model to Understand Multilingual Speech via Multi-Instructional Training
Pavel Denisov, Ngoc Thang Vu
Recent advancements in language modeling have led to the emergence of Large Language Models (LLMs) capable of various natural language processing tasks. Despite their success in te…
The IMS Toucan System for the Blizzard Challenge 2023
Florian Lux, Julia Koch, Sarina Meyer +5
For our contribution to the Blizzard Challenge 2023, we improved on the system we submitted to the Blizzard Challenge 2021. Our approach entails a rule-based text-to-phoneme proces…
Leveraging Multilingual Self-Supervised Pretrained Models for Sequence-to-Sequence End-to-End Spoken Language Understanding
Pavel Denisov, Ngoc Thang Vu
A number of methods have been proposed for End-to-End Spoken Language Understanding (E2E-SLU) using pretrained models, however their evaluation often lacks multilingual setup and t…
Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study
Xuankai Chang, Brian Yan, Kwanghee Choi +14
Speech signals, typically sampled at rates in the tens of thousands per second, contain redundancies, evoking inefficiencies in sequence modeling. High-dimensional speech features…
Anonymizing Speech with Generative Adversarial Networks to Preserve Speaker Privacy
Sarina Meyer, Pascal Tilli, Pavel Denisov +3
In order to protect the privacy of speech data, speaker anonymization aims for hiding the identity of a speaker by changing the voice in speech recordings. This typically comes wit…
Speaker Anonymization with Phonetic Intermediate Representations
Sarina Meyer, Florian Lux, Pavel Denisov +3
In this work, we propose a speaker anonymization pipeline that leverages high quality automatic speech recognition and synthesis systems to generate speech conditioned on phonetic…
ESPnet-SLU: Advancing Spoken Language Understanding through ESPnet
Siddhant Arora, Siddharth Dalmia, Pavel Denisov +10
As Automatic Speech Processing (ASR) systems are getting better, there is an increasing interest of using the ASR output to do downstream Natural Language Processing (NLP) tasks. H…
Investigations on Speech Recognition Systems for Low-Resource Dialectal Arabic-English Code-Switching Speech
Injy Hamed, Pavel Denisov, Chia-Yu Li +3
Code-switching (CS), defined as the mixing of languages in conversations, has become a worldwide phenomenon. The prevalence of CS has been recently met with a growing demand and in…
IMS' Systems for the IWSLT 2021 Low-Resource Speech Translation Task
Pavel Denisov, Manuel Mager, Ngoc Thang Vu
This paper describes the submission to the IWSLT 2021 Low-Resource Speech Translation Shared Task by IMS team. We utilize state-of-the-art models combined with several data augment…
Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis
Desh Raj, Pavel Denisov, Zhuo Chen +11
Multi-speaker speech recognition of unsegmented recordings has diverse applications such as meeting transcription and automatic subtitle generation. With technical advances in syst…
Pretrained Semantic Speech Embeddings for End-to-End Spoken Language Understanding via Cross-Modal Teacher-Student Learning
Pavel Denisov, Ngoc Thang Vu
Spoken language understanding is typically based on pipeline architectures including speech recognition and natural language understanding steps. These components are optimized ind…
ADVISER: A Toolkit for Developing Multi-modal, Multi-domain and Socially-engaged Conversational Agents
Chia-Yu Li, Daniel Ortega, Dirk Väth +9
We present ADVISER - an open-source, multi-domain dialog system toolkit that enables the development of multi-modal (incorporating speech, text and vision), socially-engaged (e.g.…
IMS-Speech: A Speech to Text Tool
Pavel Denisov, Ngoc Thang Vu
We present the IMS-Speech, a web based tool for German and English speech transcription aiming to facilitate research in various disciplines which require accesses to lexical infor…
End-to-End Multi-Speaker Speech Recognition using Speaker Embeddings and Transfer Learning
Pavel Denisov, Ngoc Thang Vu
This paper presents our latest investigation on end-to-end automatic speech recognition (ASR) for overlapped speech. We propose to train an end-to-end system conditioned on speaker…
Context-aware Neural-based Dialog Act Classification on Automatically Generated Transcriptions
Daniel Ortega, Chia-Yu Li, Gisela Vallejo +2
This paper presents our latest investigations on dialog act (DA) classification on automatically generated transcriptions. We propose a novel approach that combines convolutional n…
Unsupervised Domain Adaptation by Adversarial Learning for Robust Speech Recognition
Pavel Denisov, Ngoc Thang Vu, Marc Ferras Font
In this paper, we investigate the use of adversarial learning for unsupervised adaptation to unseen recording conditions, more specifically, single microphone far-field speech. We…