20 citations · 51 across the 21 of their papers we have counts for
22 papers
Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering
Junyu Dai, Xinyue Fan, Weiqin Li +14
In this report, we present a unified song generation framework capable of producing high-quality full-length music from lyrics, text descriptions, and musical attributes. The propo…
Cross-modal Consistency Guidance for Robust Emotion Control in Auto-Regressive TTS Models
Yizhou Peng, Yukun Ma, Chong Zhang +4
While Text-to-Speech (TTS) systems enable emotional control via natural-language instructions, expressiveness, naturalness, and speech quality degrade when the target emotion confl…
Fun-ASR Technical Report
Keyu An, Yanni Chen, Zhigao Chen +35
In recent years, automatic speech recognition (ASR) has witnessed transformative advancements driven by three complementary paradigms: data scaling, model size scaling, and deep in…
FD-Bench: A Full-Duplex Benchmarking Pipeline Designed for Full Duplex Spoken Dialogue Systems
Yizhou Peng, Yi-Wen Chao, Dianwen Ng +4
Full-duplex spoken dialogue systems (FDSDS) enable more natural human-machine interactions by allowing real-time user interruptions and backchanneling, compared to traditional SDS…
CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training
Zhihao Du, Changfeng Gao, Yuxuan Wang +19
In our prior works, we introduced a scalable streaming speech synthesis model, CosyVoice 2, which integrates a large language model (LLM) and a chunk-aware flow matching (FM) model…
InspireMusic: Integrating Super Resolution and Large Language Model for High-Fidelity Long-Form Music Generation
Chong Zhang, Yukun Ma, Qian Chen +12
We introduce InspireMusic, a framework integrated super resolution and large language model for high-fidelity long-form music generation. A unified framework generates high-fidelit…
MinMo: A Multimodal Large Language Model for Seamless Voice Interaction
Qian Chen, Yafeng Chen, Yanni Chen +33
Recent advancements in large language models (LLMs) and multimodal speech-text models have laid the groundwork for seamless voice interactions, enabling real-time, natural, and hum…
Emotional Dimension Control in Language Model-Based Text-to-Speech: Spanning a Broad Spectrum of Human Emotions
Kun Zhou, You Zhang, Dianwen Ng +3
Emotional text-to-speech (TTS) systems sturggle to capture the full spectrum of human emotions due to the inherent complexity of emotional expressions and the limited coverage of e…
FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs
Keyu An, Qian Chen, Chong Deng +30
This report introduces FunAudioLLM, a model family designed to enhance natural voice interactions between humans and large language models (LLMs). At its core are two innovative mo…
Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis
Kun Zhou, Shengkui Zhao, Yukun Ma +7
Recent language model-based text-to-speech (TTS) frameworks demonstrate scalability and in-context learning capabilities. However, they suffer from robustness issues due to the acc…
MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separation
Shengkui Zhao, Yukun Ma, Chongjia Ni +7
Our previously proposed MossFormer has achieved promising performance in monaural speech separation. However, it predominantly adopts a self-attention-based MossFormer module, whic…
Are Soft Prompts Good Zero-shot Learners for Speech Recognition?
Dianwen Ng, Chong Zhang, Ruixi Zhang +7
Large self-supervised pre-trained speech models require computationally expensive fine-tuning for downstream tasks. Soft prompt tuning offers a simple parameter-efficient alternati…
SPGM: Prioritizing Local Features for enhanced speech separation performance
Jia Qi Yip, Shengkui Zhao, Yukun Ma +8
Dual-path is a popular architecture for speech separation models (e.g. Sepformer) which splits long sequences into overlapping chunks for its intra- and inter-blocks that separatel…
ACA-Net: Towards Lightweight Speaker Verification using Asymmetric Cross Attention
Jia Qi Yip, Tuan Truong, Dianwen Ng +7
In this paper, we propose ACA-Net, a lightweight, global context-aware speaker embedding extractor for Speaker Verification (SV) that improves upon existing work by using Asymmetri…
Contrastive Speech Mixup for Low-resource Keyword Spotting
Dianwen Ng, Ruixi Zhang, Jia Qi Yip +6
Most of the existing neural-based models for keyword spotting (KWS) in smart devices require thousands of training samples to learn a decent audio representation. However, with the…
deHuBERT: Disentangling Noise in a Self-supervised Model for Robust Speech Recognition
Dianwen Ng, Ruixi Zhang, Jia Qi Yip +7
Existing self-supervised pre-trained speech models have offered an effective way to leverage massive unannotated corpora to build good automatic speech recognition (ASR). However,…
Cloud-based Automatic Speech Recognition Systems for Southeast Asian Languages
Lei Wang, Rong Tong, Cheung Chi Leung +3
This paper provides an overall introduction of our Automatic Speech Recognition (ASR) systems for Southeast Asian languages. As not much existing work has been carried out on such…
I2CR: Improving Noise Robustness on Keyword Spotting Using Inter-Intra Contrastive Regularization
Dianwen Ng, Jia Qi Yip, Tanmay Surana +6
Noise robustness in keyword spotting remains a challenge as many models fail to overcome the heavy influence of noises, causing the deterioration of the quality of feature embeddin…
A Unified Speaker Adaptation Approach for ASR
Yingzhu Zhao, Chongjia Ni, Cheung-Chi Leung +3
Transformer models have been used in automatic speech recognition (ASR) successfully and yields state-of-the-art results. However, its performance is still affected by speaker mism…
Leveraging Text Data Using Hybrid Transformer-LSTM Based End-to-End ASR in Transfer Learning
Zhiping Zeng, Van Tung Pham, Haihua Xu +4
In this work, we study leveraging extra text data to improve low-resource end-to-end ASR under cross-lingual transfer learning setting. To this end, we extend our prior work [1], a…
Independent language modeling architecture for end-to-end ASR
Van Tung Pham, Haihua Xu, Yerbolat Khassanov +5
The attention-based end-to-end (E2E) automatic speech recognition (ASR) architecture allows for joint optimization of acoustic and language models within a single network. However,…
Constrained Output Embeddings for End-to-End Code-Switching Speech Recognition with Only Monolingual Data
Yerbolat Khassanov, Haihua Xu, Van Tung Pham +4
The lack of code-switch training data is one of the major concerns in the development of end-to-end code-switching automatic speech recognition (ASR) models. In this work, we propo…