96 citations · 272 across the 20 of their papers we have counts for
20 papers
Joint Pre-Training with Speech and Bilingual Text for Direct Speech to Speech Translation
Kun Wei, Long Zhou, Ziqiang Zhang +5
Direct speech-to-speech translation (S2ST) is an attractive research topic with many advantages compared to cascaded S2ST. However, direct S2ST suffers from the data scarcity probl…
Self-supervised Context-aware Style Representation for Expressive Speech Synthesis
Yihan Wu, Xi Wang, Shaofei Zhang +3
Expressive speech synthesis, like audiobook synthesis, is still challenging for style representation learning and prediction. Deriving from reference audio or predicting style tags…
Cross-Lingual Text-to-Speech Using Multi-Task Learning and Speaker Classifier Joint Training
J. Yang, Lei He
In cross-lingual speech synthesis, the speech in various languages can be synthesized for a monoglot speaker. Normally, only the data of monoglot speakers are available for model t…
Neural Lexicon Reader: Reduce Pronunciation Errors in End-to-end TTS by Leveraging External Textual Knowledge
Mutian He, Jingzhou Yang, Lei He +1
End-to-end TTS requires a large amount of speech/text paired data to cover all necessary knowledge, particularly how to pronounce different words in diverse contexts, so that a neu…
Cross-speaker Style Transfer with Prosody Bottleneck in Neural Speech Synthesis
Shifeng Pan, Lei He
Cross-speaker style transfer is crucial to the applications of multi-style and expressive speech synthesis at scale. It does not require the target speakers to be experts in expres…
Speech BERT Embedding For Improving Prosody in Neural TTS
Liping Chen, Yan Deng, Xi Wang +2
This paper presents a speech BERT model to extract embedded prosody information in speech segments for improving the prosody of synthesized speech in neural text-to-speech (TTS). A…
On Addressing Practical Challenges for RNN-Transducer
Rui Zhao, Jian Xue, Jinyu Li +3
In this paper, several works are proposed to address practical challenges for deploying RNN Transducer (RNN-T) based speech recognition system. These challenges are adapting a well…
Exploring Machine Speech Chain for Domain Adaptation and Few-Shot Speaker Adaptation
Fengpeng Yue, Yan Deng, Lei He +1
Machine Speech Chain, which integrates both end-to-end (E2E) automatic speech recognition (ASR) and text-to-speech (TTS) into one circle for joint training, has been proven to be e…
Multilingual Byte2Speech Models for Scalable Low-resource Speech Synthesis
Mutian He, Jingzhou Yang, Lei He +1
To scale neural speech synthesis to various real-world languages, we present a multilingual end-to-end framework that maps byte inputs to spectrograms, thus allowing arbitrary inpu…
s-Transformer: Segment-Transformer for Robust Neural Speech Synthesis
Xi Wang, Huaiping Ming, Lei He +1
Neural end-to-end text-to-speech (TTS) , which adopts either a recurrent model, e.g. Tacotron, or an attention one, e.g. Transformer, to characterize a speech utterance, has achiev…
Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability
Jinyu Li, Rui Zhao, Zhong Meng +8
Because of its streaming nature, recurrent neural network transducer (RNN-T) is a very promising end-to-end (E2E) model that may replace the popular hybrid model for automatic spee…
Conversational End-to-End TTS for Voice Agent
Haohan Guo, Shaofei Zhang, Frank K. Soong +2
End-to-end neural TTS has achieved superior performance on reading style speech synthesis. However, it's still a challenge to build a high-quality conversational TTS due to the lim…
A New GAN-based End-to-End TTS Training Algorithm
Haohan Guo, Frank K. Soong, Lei He +1
End-to-end, autoregressive model-based TTS has shown significant performance improvements over the conventional one. However, the autoregressive module training is affected by the…
Exploiting Syntactic Features in a Parsed Tree to Improve End-to-End TTS
Haohan Guo, Frank K. Soong, Lei He +1
The end-to-end TTS, which can predict speech directly from a given sequence of graphemes or phonemes, has shown improved performance over the conventional TTS. However, its predict…
Feature reinforcement with word embedding and parsing information in neural TTS
Huaiping Ming, Lei He, Haohan Guo +1
In this paper, we propose a feature reinforcement method under the sequence-to-sequence neural text-to-speech (TTS) synthesis framework. The proposed method utilizes the multiple i…
Recurrent Neural Networks with Pre-trained Language Model Embedding for Slot Filling Task
Liang Qiu, Yuanyi Ding, Lei He
In recent years, Recurrent Neural Networks (RNNs) based models have been applied to the Slot Filling problem of Spoken Language Understanding and achieved the state-of-the-art perf…
Modeling Multi-speaker Latent Space to Improve Neural TTS: Quick Enrolling New Speaker and Enhancing Premium Voice
Yan Deng, Lei He, Frank Soong
Neural TTS has shown it can generate high quality synthesized speech. In this paper, we investigate the multi-speaker latent space to improve neural TTS for adapting the system to…
Learning latent representations for style control and transfer in end-to-end speech synthesis
Ya-Jie Zhang, Shifeng Pan, Lei He +1
In this paper, we introduce the Variational Autoencoder (VAE) to an end-to-end speech synthesis model, to learn the latent representation of speaking styles in an unsupervised mann…
A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding
Peilu Wang, Yao Qian, Frank K. Soong +2
Bidirectional Long Short-Term Memory Recurrent Neural Network (BLSTM-RNN) has been shown to be very effective for modeling and predicting sequential data, e.g. speech utterances or…
Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Recurrent Neural Network
Peilu Wang, Yao Qian, Frank K. Soong +2
Bidirectional Long Short-Term Memory Recurrent Neural Network (BLSTM-RNN) has been shown to be very effective for tagging sequential data, e.g. speech utterances or handwritten doc…