9 citations · 30 across the 18 of their papers we have counts for
18 papers
Sample-Efficient Diffusion for Text-To-Speech Synthesis
Justin Lovelace, Soham Ray, Kwangyoun Kim +2
This work introduces Sample-Efficient Speech Diffusion (SESD), an algorithm for effective speech synthesis in modest data regimes through latent diffusion. It is based on a novel d…
DiscreteSLU: A Large Language Model with Self-Supervised Discrete Speech Units for Spoken Language Understanding
Suwon Shon, Kwangyoun Kim, Yi-Te Hsu +3
The integration of pre-trained text-based large language models (LLM) with speech input has enabled instruction-following capabilities for diverse speech tasks. This integration re…
Improving ASR Contextual Biasing with Guided Attention
Jiyang Tang, Kwangyoun Kim, Suwon Shon +3
In this paper, we propose a Guided Attention (GA) auxiliary training loss, which improves the effectiveness and robustness of automatic speech recognition (ASR) contextual biasing…
Generative Context-aware Fine-tuning of Self-supervised Speech Models
Suwon Shon, Kwangyoun Kim, Prashant Sridhar +3
When performing tasks like automatic speech recognition or spoken language understanding for a given utterance, access to preceding text or audio provides contextual information ca…
A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Translation, and Understanding Tasks
Yifan Peng, Kwangyoun Kim, Felix Wu +7
Conformer, a convolution-augmented Transformer variant, has become the de facto encoder architecture for speech processing due to its superior performance in various tasks, includi…
Structured Pruning of Self-Supervised Pre-trained Models for Speech Recognition and Understanding
Yifan Peng, Kwangyoun Kim, Felix Wu +2
Self-supervised speech representation learning (SSL) has shown to be effective in various downstream tasks, but SSL models are usually large and slow. Model compression techniques…
Context-aware Fine-tuning of Self-supervised Speech Models
Suwon Shon, Felix Wu, Kwangyoun Kim +3
Self-supervised pre-trained transformers have improved the state of the art on a variety of speech tasks. Due to the quadratic time and space complexity of self-attention, they usu…
E-Branchformer: Branchformer with Enhanced merging for speech recognition
Kwangyoun Kim, Felix Wu, Yifan Peng +4
Conformer, combining convolution and self-attention sequentially to capture both local and global information, has shown remarkable performance and is currently regarded as the sta…
Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages
Felix Wu, Kwangyoun Kim, Shinji Watanabe +4
We introduce Wav2Seq, the first self-supervised approach to pre-train both parts of encoder-decoder models for speech data. We induce a pseudo language as a compact discrete repres…
SRU++: Pioneering Fast Recurrence with Attention for Speech Recognition
Jing Pan, Tao Lei, Kwangyoun Kim +2
The Transformer architecture has been well adopted as a dominant architecture in most sequence transduction tasks including automatic speech recognition (ASR), since its attention…
Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition
Felix Wu, Kwangyoun Kim, Jing Pan +3
This paper is a study of performance-efficiency trade-offs in pre-trained models for automatic speech recognition (ASR). We focus on wav2vec 2.0, and formalize several architecture…
Multi-mode Transformer Transducer with Stochastic Future Context
Kwangyoun Kim, Felix Wu, Prashant Sridhar +2
Automatic speech recognition (ASR) models make fewer errors when more surrounding speech information is presented as context. Unfortunately, acquiring a larger future context leads…
Sequential Routing Framework: Fully Capsule Network-based Speech Recognition
Kyungmin Lee, Hyunwhan Joe, Hyeontaek Lim +4
Capsule networks (CapsNets) have recently gotten attention as a novel neural architecture. This paper presents the sequential routing framework which we believe is the first method…
Small energy masking for improved neural network training for end-to-end speech recognition
Chanwoo Kim, Kwangyoun Kim, Sathish Reddy Indurthi
In this paper, we present a Small Energy Masking (SEM) algorithm, which masks inputs having values below a certain threshold. More specifically, a time-frequency bin is masked if t…
Attention based on-device streaming speech recognition with large speech corpus
Kwangyoun Kim, Kyungmin Lee, Dhananjaya Gowda +10
In this paper, we present a new on-device automatic speech recognition (ASR) system based on monotonic chunk-wise attention (MoChA) models trained with large (> 10K hours) corpus.…
Improved Multi-Stage Training of Online Attention-based Encoder-Decoder Models
Abhinav Garg, Dhananjaya Gowda, Ankur Kumar +3
In this paper, we propose a refined multi-stage multi-task training strategy to improve the performance of online attention-based encoder-decoder (AED) models. A three-stage traini…
power-law nonlinearity with maximally uniform distribution criterion for improved neural network training in automatic speech recognition
Chanwoo Kim, Mehul Kumar, Kwangyoun Kim +1
In this paper, we describe the Maximum Uniformity of Distribution (MUD) algorithm with the power-law nonlinearity. In this approach, we hypothesize that neural network training wil…
end-to-end training of a large vocabulary end-to-end speech recognition system
Chanwoo Kim, Sungsoo Kim, Kwangyoun Kim +10
In this paper, we present an end-to-end training framework for building state-of-the-art end-to-end speech recognition systems. Our training system utilizes a cluster of Central Pr…