18 citations · 76 across the 25 of their papers we have counts for
21 papers · 1 filter
Knowledge-Decoupled Functionally Invariant Path with Synthetic Personal Data for Personalized ASR
Yue Gu, Zhihao Du, Ying Shi +2
Fine-tuning generic ASR models with large-scale synthetic personal data can enhance the personalization of ASR models, but it introduces challenges in adapting to synthetic persona…
Differentiable Reward Optimization for LLM based TTS system
Changfeng Gao, Zhihao Du, Shiliang Zhang
This paper proposes a novel Differentiable Reward Optimization (DiffRO) method aimed at enhancing the performance of neural codec language models based text-to-speech (TTS) systems…
SyncSpeech: Efficient and Low-Latency Text-to-Speech based on Temporal Masked Transformer
Zhengyan Sheng, Zhihao Du, Shiliang Zhang +2
Current text-to-speech (TTS) models face a persistent limitation: autoregressive (AR) models suffer from low generation efficiency, while modern non-autoregressive (NAR) models exp…
FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs
Keyu An, Qian Chen, Chong Deng +30
This report introduces FunAudioLLM, a model family designed to enhance natural voice interactions between humans and large language models (LLMs). At its core are two innovative mo…
CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens
Zhihao Du, Qian Chen, Shiliang Zhang +9
Recent years have witnessed a trend that large language model (LLM) based text-to-speech (TTS) emerges into the mainstream due to their high naturalness and zero-shot capacity. In…
SA-Paraformer: Non-autoregressive End-to-End Speaker-Attributed ASR
Yangze Li, Fan Yu, Yuhao Liang +5
Joint modeling of multi-speaker ASR and speaker diarization has recently shown promising results in speaker-attributed automatic speech recognition (SA-ASR).Although being able to…
FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codec
Zhihao Du, Shiliang Zhang, Kai Hu +1
This paper presents FunCodec, a fundamental neural speech codec toolkit, which is an extension of the open-source speech processing toolkit FunASR. FunCodec provides reproducible t…
The second multi-channel multi-party meeting transcription challenge (M2MeT) 2.0): A benchmark for speaker-attributed ASR
Yuhao Liang, Mohan Shi, Fan Yu +11
With the success of the first Multi-channel Multi-party Meeting Transcription challenge (M2MeT), the second M2MeT challenge (M2MeT 2.0) held in ASRU2023 particularly aims to tackle…
LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT
Zhihao Du, Jiaming Wang, Qian Chen +12
Generative Pre-trained Transformer (GPT) models have achieved remarkable performance on various natural language processing tasks, and have shown great potential as backbones for a…
FunASR: A Fundamental End-to-End Speech Recognition Toolkit
Zhifu Gao, Zerui Li, Jiaming Wang +8
This paper introduces FunASR, an open-source speech recognition toolkit designed to bridge the gap between academic research and industrial applications. FunASR offers models train…
TOLD: A Novel Two-Stage Overlap-Aware Framework for Speaker Diarization
Jiaming Wang, Zhihao Du, Shiliang Zhang
Recently, end-to-end neural diarization (EEND) is introduced and achieves promising results in speaker-overlapped scenarios. In EEND, speaker diarization is formulated as a multi-l…
Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis
Zhihao Du, Shiliang Zhang, Siqi Zheng +1
Recently, hybrid systems of clustering and neural diarization models have been successfully applied in multi-party meeting analysis. However, current models always treat overlapped…
MFCCA:Multi-Frame Cross-Channel attention for multi-speaker ASR in Multi-party meeting scenario
Fan Yu, Shiliang Zhang, Pengcheng Guo +4
Recently cross-channel attention, which better leverages multi-channel signals from microphone array, has shown promising results in the multi-party meeting scenario. Cross-channel…
Speaker Embedding-aware Neural Diarization: an Efficient Framework for Overlapping Speech Diarization in Meeting Scenarios
Zhihao Du, Shiliang Zhang, Siqi Zheng +1
Overlapping speech diarization has been traditionally treated as a multi-label classification problem. In this paper, we reformulate this task as a single-label prediction problem…
A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings
Fan Yu, Zhihao Du, Shiliang Zhang +2
In this paper, we conduct a comparative study on speaker-attributed automatic speech recognition (SA-ASR) in the multi-party meeting scenario, a topic with increasing attention in…
Summary On The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Grand Challenge
Fan Yu, Shiliang Zhang, Pengcheng Guo +13
The ICASSP 2022 Multi-channel Multi-party Meeting Transcription Grand Challenge (M2MeT) focuses on one of the most valuable and the most challenging scenarios of speech technologie…
Speaker Embedding-aware Neural Diarization for Flexible Number of Speakers with Textual Information
Zhihao Du, Shiliang Zhang, Siqi Zheng +2
Overlapping speech diarization is always treated as a multi-label classification problem. In this paper, we reformulate this task as a single-label prediction problem by encoding t…
M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge
Fan Yu, Shiliang Zhang, Yihui Fu +9
Recent development of speech processing, such as speech recognition, speaker diarization, etc., has inspired numerous applications of speech technologies. The meeting scenario is o…
A Monaural Speech Enhancement Method for Robust Small-Footprint Keyword Spotting
Yue Gu, Zhihao Du, Hui Zhang +1
Robustness against noise is critical for keyword spotting (KWS) in real-world environments. To improve the robustness, a speech enhancement front-end is involved. Instead of treati…
Acoustic Scene Classification by Implicitly Identifying Distinct Sound Events
Hongwei Song, Jiqing Han, Shiwen Deng +1
In this paper, we propose a new strategy for acoustic scene classification (ASC) , namely recognizing acoustic scenes through identifying distinct sound events. This differs from e…
Investigation of Monaural Front-End Processing for Robust ASR without Retraining or Joint-Training
Zhihao Du, Xueliang Zhang, Jiqing Han
In recent years, monaural speech separation has been formulated as a supervised learning problem, which has been systematically researched and shown the dramatical improvement of s…