34 citations · 195 across the 59 of their papers we have counts for
19 papers · 1 filter
PolyVoice: Language Models for Speech to Speech Translation
Qianqian Dong, Zhiying Huang, Qiao Tian +15
We propose PolyVoice, a language model-based framework for speech-to-speech translation (S2ST) system. Our framework consists of two language models: a translation language model a…
Text-only Domain Adaptation using Unified Speech-Text Representation in Transducer
Lu Huang, Boyu Li, Jun Zhang +2
Domain adaptation using text-only corpus is challenging in end-to-end(E2E) speech recognition. Adaptation by synthesizing audio from text through TTS is resource-consuming. We pres…
CIF-PT: Bridging Speech and Text Representations for Spoken Language Understanding via Continuous Integrate-and-Fire Pre-Training
Linhao Dong, Zhecheng An, Peihao Wu +3
Speech or text representation generated by pre-trained models contains modal-specific information that could be combined for benefiting spoken language understanding (SLU) tasks. I…
Phonetic and Prosody-aware Self-supervised Learning Approach for Non-native Fluency Scoring
Kaiqi Fu, Shaojun Gao, Shuju Shi +3
Speech fluency/disfluency can be evaluated by analyzing a range of phonetic and prosodic features. Deep neural networks are commonly trained to map fluency-related features into th…
SCM: Enhancing Large Language Model with Self-Controlled Memory Framework
Bing Wang, Xinnian Liang, Jian Yang +6
Large Language Models (LLMs) are constrained by their inability to process lengthy inputs, resulting in the loss of critical historical information. To address this limitation, in…
LiteG2P: A fast, light and high accuracy model for grapheme-to-phoneme conversion
Chunfeng Wang, Peisong Huang, Yuxiang Zou +4
As a key component of automated speech recognition (ASR) and the front-end in text-to-speech (TTS), grapheme-to-phoneme (G2P) plays the role of converting letters to their correspo…
Direct Speech-to-speech Translation without Textual Annotation using Bottleneck Features
Junhui Zhang, Junjie Pan, Xiang Yin +1
Speech-to-speech translation directly translates a speech utterance to another between different languages, and has great potential in tasks such as simultaneous interpretation. St…
BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performance
Haotong Qin, Xudong Ma, Yifu Ding +6
Deep neural networks, such as the Deep-FSMN, have been widely studied for keyword spotting (KWS) applications while suffering expensive computation and storage. Therefore, network…
Internal Language Model Estimation based Adaptive Language Model Fusion for Domain Adaptation
Rao Ma, Xiaobo Wu, Jin Qiu +4
ASR model deployment environment is ever-changing, and the incoming speech can be switched across different domains during a session. This brings a challenge for effective domain a…
A Novel Chinese Dialect TTS Frontend with Non-Autoregressive Neural Machine Translation
Junhui Zhang, Wudi Bao, Junjie Pan +2
Chinese dialects are different variations of Chinese and can be considered as different languages in the same language family with Mandarin. Though they all use Chinese characters,…
Improving Contextual Representation with Gloss Regularized Pre-training
Yu Lin, Zhecheng An, Peihao Wu +1
Though achieving impressive results on many NLP tasks, the BERT-like masked language models (MLM) encounter the discrepancy between pre-training and inference. In light of this gap…
Improving End-to-End Contextual Speech Recognition with Fine-Grained Contextual Knowledge Selection
Minglun Han, Linhao Dong, Zhenlin Liang +4
Nowadays, most methods in end-to-end contextual speech recognition bias the recognition process towards contextual knowledge. Since all-neural contextual biasing methods rely on ph…
BiFSMN: Binary Neural Network for Keyword Spotting
Haotong Qin, Xudong Ma, Yifu Ding +6
The deep neural networks, such as the Deep-FSMN, have been widely studied for keyword spotting (KWS) applications. However, computational resources for these networks are significa…
Improving Accent Conversion with Reference Encoder and End-To-End Text-To-Speech
Wenjie Li, Benlai Tang, Xiang Yin +6
Accent conversion (AC) transforms a non-native speaker's accent into a native accent while maintaining the speaker's voice timbre. In this paper, we propose approaches to improving…
A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesis
Junjie Pan, Xiang Yin, Zhiling Zhang +4
In Mandarin text-to-speech (TTS) system, the front-end text processing module significantly influences the intelligibility and naturalness of synthesized speech. Building a typical…
A hybrid text normalization system using multi-head self-attention for mandarin
Junhui Zhang, Junjie Pan, Xiang Yin +5
In this paper, we propose a hybrid text normalization system using multi-head self-attention. The system combines the advantages of a rule-based model and a neural model for text p…
Frame Stacking and Retaining for Recurrent Neural Network Acoustic Model
Xu Tian, Jun Zhang, Zejun Ma +2
Frame stacking is broadly applied in end-to-end neural network training like connectionist temporal classification (CTC), and it leads to more accurate models and faster decoding.…
Deep LSTM for Large Vocabulary Continuous Speech Recognition
Xu Tian, Jun Zhang, Zejun Ma +6
Recurrent neural networks (RNNs), especially long short-term memory (LSTM) RNNs, are effective network for sequential task like speech recognition. Deeper LSTM models perform well…
Exponential Moving Average Model in Parallel Speech Recognition Training
Xu Tian, Jun Zhang, Zejun Ma +2
As training data rapid growth, large-scale parallel training with multi-GPUs cluster is widely applied in the neural network model learning currently.We present a new approach that…