11 citations · 30 across the 15 of their papers we have counts for
10 papers · 1 filter
Teaching a Multilingual Large Language Model to Understand Multilingual Speech via Multi-Instructional Training
Pavel Denisov, Ngoc Thang Vu
Recent advancements in language modeling have led to the emergence of Large Language Models (LLMs) capable of various natural language processing tasks. Despite their success in te…
The IMS Toucan System for the Blizzard Challenge 2023
Florian Lux, Julia Koch, Sarina Meyer +5
For our contribution to the Blizzard Challenge 2023, we improved on the system we submitted to the Blizzard Challenge 2021. Our approach entails a rule-based text-to-phoneme proces…
Leveraging Multilingual Self-Supervised Pretrained Models for Sequence-to-Sequence End-to-End Spoken Language Understanding
Pavel Denisov, Ngoc Thang Vu
A number of methods have been proposed for End-to-End Spoken Language Understanding (E2E-SLU) using pretrained models, however their evaluation often lacks multilingual setup and t…
Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study
Xuankai Chang, Brian Yan, Kwanghee Choi +14
Speech signals, typically sampled at rates in the tens of thousands per second, contain redundancies, evoking inefficiencies in sequence modeling. High-dimensional speech features…
ESPnet-SLU: Advancing Spoken Language Understanding through ESPnet
Siddhant Arora, Siddharth Dalmia, Pavel Denisov +10
As Automatic Speech Processing (ASR) systems are getting better, there is an increasing interest of using the ASR output to do downstream Natural Language Processing (NLP) tasks. H…
Investigations on Speech Recognition Systems for Low-Resource Dialectal Arabic-English Code-Switching Speech
Injy Hamed, Pavel Denisov, Chia-Yu Li +3
Code-switching (CS), defined as the mixing of languages in conversations, has become a worldwide phenomenon. The prevalence of CS has been recently met with a growing demand and in…
IMS' Systems for the IWSLT 2021 Low-Resource Speech Translation Task
Pavel Denisov, Manuel Mager, Ngoc Thang Vu
This paper describes the submission to the IWSLT 2021 Low-Resource Speech Translation Shared Task by IMS team. We utilize state-of-the-art models combined with several data augment…
ADVISER: A Toolkit for Developing Multi-modal, Multi-domain and Socially-engaged Conversational Agents
Chia-Yu Li, Daniel Ortega, Dirk Väth +9
We present ADVISER - an open-source, multi-domain dialog system toolkit that enables the development of multi-modal (incorporating speech, text and vision), socially-engaged (e.g.…
IMS-Speech: A Speech to Text Tool
Pavel Denisov, Ngoc Thang Vu
We present the IMS-Speech, a web based tool for German and English speech transcription aiming to facilitate research in various disciplines which require accesses to lexical infor…
Context-aware Neural-based Dialog Act Classification on Automatically Generated Transcriptions
Daniel Ortega, Chia-Yu Li, Gisela Vallejo +2
This paper presents our latest investigations on dialog act (DA) classification on automatically generated transcriptions. We propose a novel approach that combines convolutional n…