85 citations · 288 across the 86 of their papers we have counts for
20 papers · 1 filter
SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codec
Chunyu Qiang, Haoyu Wang, Cheng Gong +10
Speech codecs serve as a crucial bridge in unifying speech and text language models. Existing codec methods face several challenges in semantic encoding, such as residual paralingu…
Text Prompt is Not Enough: Sound Event Enhanced Prompt Adapter for Target Style Audio Generation
Chenxu Xiong, Ruibo Fu, Shuchen Shi +9
Current mainstream audio generation methods primarily rely on simple text prompts, often failing to capture the nuanced details necessary for multi-style audio generation. To addre…
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing
Chunyu Qiang, Wang Geng, Yi Zhao +12
Deep learning has brought significant improvements to the field of cross-modal representation learning. For tasks such as text-to-speech (TTS), voice conversion (VC), and automatic…
ICAGC 2024: Inspirational and Convincing Audio Generation Challenge 2024
Ruibo Fu, Rui Liu, Chunyu Qiang +11
The Inspirational and Convincing Audio Generation Challenge 2024 (ICAGC 2024) is part of the ISCSLP 2024 Competitions and Challenges track. While current text-to-speech (TTS) techn…
ASRRL-TTS: Agile Speaker Representation Reinforcement Learning for Text-to-Speech Speaker Adaptation
Ruibo Fu, Xin Qi, Zhengqi Wen +10
Speaker adaptation, which involves cloning voices from unseen speakers in the Text-to-Speech task, has garnered significant interest due to its numerous applications in multi-media…
MINT: a Multi-modal Image and Narrative Text Dubbing Dataset for Foley Audio Content Planning and Generation
Ruibo Fu, Shuchen Shi, Hongming Guo +12
Foley audio, critical for enhancing the immersive experience in multimedia content, faces significant challenges in the AI-generated content (AIGC) landscape. Despite advancements…
FSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularization
Zhengkun Tian, Jiangyan Yi, Ye Bai +3
Transducer-based models, such as RNN-Transducer and transformer-transducer, have achieved great success in speech recognition. A typical transducer model decodes the output sequenc…
TSNAT: Two-Step Non-Autoregressvie Transformer Models for Speech Recognition
Zhengkun Tian, Jiangyan Yi, Jianhua Tao +4
The autoregressive (AR) models, such as attention-based encoder-decoder models and RNN-Transducer, have achieved great success in speech recognition. They predict the output sequen…
Latent linguistic embedding for cross-lingual text-to-speech and voice conversion
Hieu-Thi Luong, Junichi Yamagishi
As the recently proposed voice cloning system, NAUTILUS, is capable of cloning unseen voices using untranscribed speech, we investigate the feasibility of using it to develop a uni…
One In A Hundred: Select The Best Predicted Sequence from Numerous Candidates for Streaming Speech Recognition
Zhengkun Tian, Jiangyan Yi, Ye Bai +3
The RNN-Transducers and improved attention-based encoder-decoder models are widely applied to streaming speech recognition. Compared with these two end-to-end models, the CTC model…
Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognition
Zhengkun Tian, Jiangyan Yi, Jianhua Tao +3
Non-autoregressive transformer models have achieved extremely fast inference speed and comparable performance with autoregressive sequence-to-sequence models in neural machine tran…
Listen Attentively, and Spell Once: Whole Sentence Generation via a Non-Autoregressive Architecture for Low-Latency Speech Recognition
Ye Bai, Jiangyan Yi, Jianhua Tao +3
Although attention based end-to-end models have achieved promising performance in speech recognition, the multi-pass forward computation in beam-search increases inference time cos…
Simultaneous Denoising and Dereverberation Using Deep Embedding Features
Cunhang Fan, Jianhua Tao, Bin Liu +2
Monaural speech dereverberation is a very challenging task because no spatial cues can be used. When the additive noises exist, this task becomes more challenging. In this paper, w…
Deep Attention Fusion Feature for Speech Separation with End-to-End Post-filter Method
Cunhang Fan, Jianhua Tao, Bin Liu +3
In this paper, we propose an end-to-end post-filter method with deep attention fusion features for monaural speaker-independent speech separation. At first, a time-frequency domain…
Spatial and spectral deep attention fusion for multi-channel speech separation using deep embedding features
Cunhang Fan, Bin Liu, Jianhua Tao +2
Multi-channel deep clustering (MDC) has acquired a good performance for speech separation. However, MDC only applies the spatial features as the additional information. So it is di…
Synchronous Transformers for End-to-End Speech Recognition
Zhengkun Tian, Jiangyan Yi, Ye Bai +3
For most of the attention-based sequence-to-sequence models, the decoder predicts the output sequence conditioned on the entire input sequence processed by the encoder. The asynchr…
Integrating Knowledge into End-to-End Speech Recognition from External Text-Only Data
Ye Bai, Jiangyan Yi, Jianhua Tao +3
Attention-based encoder-decoder (AED) models have achieved promising performance in speech recognition. However, because of the end-to-end training, an AED model is usually trained…
Self-Attention Transducers for End-to-End Speech Recognition
Zhengkun Tian, Jiangyan Yi, Jianhua Tao +2
Recurrent neural network transducers (RNN-T) have been successfully applied in end-to-end speech recognition. However, the recurrent structure makes it difficult for parallelizatio…
Forward-Backward Decoding for Regularizing End-to-End TTS
Yibin Zheng, Xi Wang, Lei He +4
Neural end-to-end TTS can generate very high-quality synthesized speech, and even close to human recording within similar domain text. However, it performs unsatisfactory when scal…
Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition
Ye Bai, Jiangyan Yi, Jianhua Tao +2
Integrating an external language model into a sequence-to-sequence speech recognition system is non-trivial. Previous works utilize linear interpolation or a fusion network to inte…