9 citations · 30 across the 18 of their papers we have counts for
10 papers · 1 filter
Context-aware Fine-tuning of Self-supervised Speech Models
Suwon Shon, Felix Wu, Kwangyoun Kim +3
Self-supervised pre-trained transformers have improved the state of the art on a variety of speech tasks. Due to the quadratic time and space complexity of self-attention, they usu…
E-Branchformer: Branchformer with Enhanced merging for speech recognition
Kwangyoun Kim, Felix Wu, Yifan Peng +4
Conformer, combining convolution and self-attention sequentially to capture both local and global information, has shown remarkable performance and is currently regarded as the sta…
SRU++: Pioneering Fast Recurrence with Attention for Speech Recognition
Jing Pan, Tao Lei, Kwangyoun Kim +2
The Transformer architecture has been well adopted as a dominant architecture in most sequence transduction tasks including automatic speech recognition (ASR), since its attention…
Multi-mode Transformer Transducer with Stochastic Future Context
Kwangyoun Kim, Felix Wu, Prashant Sridhar +2
Automatic speech recognition (ASR) models make fewer errors when more surrounding speech information is presented as context. Unfortunately, acquiring a larger future context leads…
Sequential Routing Framework: Fully Capsule Network-based Speech Recognition
Kyungmin Lee, Hyunwhan Joe, Hyeontaek Lim +4
Capsule networks (CapsNets) have recently gotten attention as a novel neural architecture. This paper presents the sequential routing framework which we believe is the first method…
Small energy masking for improved neural network training for end-to-end speech recognition
Chanwoo Kim, Kwangyoun Kim, Sathish Reddy Indurthi
In this paper, we present a Small Energy Masking (SEM) algorithm, which masks inputs having values below a certain threshold. More specifically, a time-frequency bin is masked if t…
Attention based on-device streaming speech recognition with large speech corpus
Kwangyoun Kim, Kyungmin Lee, Dhananjaya Gowda +10
In this paper, we present a new on-device automatic speech recognition (ASR) system based on monotonic chunk-wise attention (MoChA) models trained with large (> 10K hours) corpus.…
Improved Multi-Stage Training of Online Attention-based Encoder-Decoder Models
Abhinav Garg, Dhananjaya Gowda, Ankur Kumar +3
In this paper, we propose a refined multi-stage multi-task training strategy to improve the performance of online attention-based encoder-decoder (AED) models. A three-stage traini…
power-law nonlinearity with maximally uniform distribution criterion for improved neural network training in automatic speech recognition
Chanwoo Kim, Mehul Kumar, Kwangyoun Kim +1
In this paper, we describe the Maximum Uniformity of Distribution (MUD) algorithm with the power-law nonlinearity. In this approach, we hypothesize that neural network training wil…
end-to-end training of a large vocabulary end-to-end speech recognition system
Chanwoo Kim, Sungsoo Kim, Kwangyoun Kim +10
In this paper, we present an end-to-end training framework for building state-of-the-art end-to-end speech recognition systems. Our training system utilizes a cluster of Central Pr…