87 citations · 163 across the 34 of their papers we have counts for
36 papers
Progressive Residual Extraction based Pre-training for Speech Representation Learning
Tianrui Wang, Jin Li, Ziyang Ma +8
Self-supervised learning (SSL) has garnered significant attention in speech processing, excelling in linguistic tasks such as speech recognition. However, jointly improving the per…
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing
Chunyu Qiang, Wang Geng, Yi Zhao +12
Deep learning has brought significant improvements to the field of cross-modal representation learning. For tasks such as text-to-speech (TTS), voice conversion (VC), and automatic…
Error Correction by Paying Attention to Both Acoustic and Confidence References for Automatic Speech Recognition
Yuchun Shu, Bo Hu, Yifeng He +3
Accurately finding the wrong words in the automatic speech recognition (ASR) hypothesis and recovering them well-founded is the goal of speech error correction. In this paper, we p…
A Refining Underlying Information Framework for Monaural Speech Enhancement
Rui Cao, Tianrui Wang, Meng Ge +2
Supervised speech enhancement has gained significantly from recent advancements in neural networks, especially due to their ability to non-linearly fit the diverse representations…
Multi-Level Knowledge Distillation for Speech Emotion Recognition in Noisy Conditions
Yang Liu, Haoqin Sun, Geng Chen +4
Speech emotion recognition (SER) performance deteriorates significantly in the presence of noise, making it challenging to achieve competitive performance in noisy conditions. To t…
ZMM-TTS: Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representations
Cheng Gong, Xin Wang, Erica Cooper +5
Neural text-to-speech (TTS) has achieved human-like synthetic speech for single-speaker, single-language synthesis. Multilingual TTS systems are limited to resource-rich languages…
High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models
Chunyu Qiang, Hao Li, Yixin Tian +4
Text-to-speech (TTS) methods have shown promising results in voice cloning, but they require a large number of labeled text-speech pairs. Minimally-supervised speech synthesis deco…
Learning Speech Representation From Contrastive Token-Acoustic Pretraining
Chunyu Qiang, Hao Li, Yixin Tian +4
For fine-grained generation and recognition tasks such as minimally-supervised text-to-speech (TTS), voice conversion (VC), and automatic speech recognition (ASR), the intermediate…
Minimally-Supervised Speech Synthesis with Conditional Diffusion Model and Language Model: A Comparative Study of Semantic Coding
Chunyu Qiang, Hao Li, Hao Ni +5
Recently, there has been a growing interest in text-to-speech (TTS) methods that can be trained with minimal supervision by combining two types of discrete speech representations a…
Rethinking the visual cues in audio-visual speaker extraction
Junjie Li, Meng Ge, Zexu pan +4
The Audio-Visual Speaker Extraction (AVSE) algorithm employs parallel video recording to leverage two visual cues, namely speaker identity and synchronization, to enhance performan…
Locate and Beamform: Two-dimensional Locating All-neural Beamformer for Multi-channel Speech Separation
Yanjie Fu, Meng Ge, Honglong Wang +7
Recently, stunning improvements on multi-channel speech separation have been achieved by neural beamformers when direction information is available. However, most of them neglect t…
speech and noise dual-stream spectrogram refine network with speech distortion loss for robust speech recognition
Haoyu Lu, Nan Li, Tongtong Song +4
In recent years, the joint training of speech enhancement front-end and automatic speech recognition (ASR) back-end has been widely used to improve the robustness of ASR systems. T…
Time-domain Speech Enhancement Assisted by Multi-resolution Frequency Encoder and Decoder
Hao Shi, Masato Mimura, Longbiao Wang +2
Time-domain speech enhancement (SE) has recently been intensively investigated. Among recent works, DEMUCS introduces multi-resolution STFT loss to enhance performance. However, so…
Cross-modal Audio-visual Co-learning for Text-independent Speaker Verification
Meng Liu, Kong Aik Lee, Longbiao Wang +3
Visual speech (i.e., lip motion) is highly related to auditory speech due to the co-occurrence and synchronization in speech production. This paper investigates this correlation an…
MIMO-DBnet: Multi-channel Input and Multiple Outputs DOA-aware Beamforming Network for Speech Separation
Yanjie Fu, Haoran Yin, Meng Ge +5
Recently, many deep learning based beamformers have been proposed for multi-channel speech separation. Nevertheless, most of them rely on extra cues known in advance, such as speak…
The ISCSLP 2022 Intelligent Cockpit Speech Recognition Challenge (ICSRC): Dataset, Tracks, Baseline and Results
Ao Zhang, Fan Yu, Kaixun Huang +7
This paper summarizes the outcomes from the ISCSLP 2022 Intelligent Cockpit Speech Recognition Challenge (ICSRC). We first address the necessity of the challenge and then introduce…
I4U System Description for NIST SRE'20 CTS Challenge
Kong Aik Lee, Tomi Kinnunen, Daniele Colibro +23
This manuscript describes the I4U submission to the 2020 NIST Speaker Recognition Evaluation (SRE'20) Conversational Telephone Speech (CTS) Challenge. The I4U's submission was resu…
Monolingual Recognizers Fusion for Code-switching Speech Recognition
Tongtong Song, Qiang Xu, Haoyu Lu +5
The bi-encoder structure has been intensively investigated in code-switching (CS) automatic speech recognition (ASR). However, most existing methods require the structures of two m…
Deep Spectro-temporal Artifacts for Detecting Synthesized Speech
Xiaohui Liu, Meng Liu, Lin Zhang +7
The Audio Deep Synthesis Detection (ADD) Challenge has been held to detect generated human-like speech. With our submitted system, this paper provides an overall assessment of trac…
VCSE: Time-Domain Visual-Contextual Speaker Extraction Network
Junjie Li, Meng Ge, Zexu Pan +2
Speaker extraction seeks to extract the target speech in a multi-talker scenario given an auxiliary reference. Such reference can be auditory, i.e., a pre-recorded speech, visual,…
Language-specific Characteristic Assistance for Code-switching Speech Recognition
Tongtong Song, Qiang Xu, Meng Ge +5
Dual-encoder structure successfully utilizes two language-specific encoders (LSEs) for code-switching speech recognition. Because LSEs are initialized by two pre-trained language-s…
MIMO-DoAnet: Multi-channel Input and Multiple Outputs DoA Network with Unknown Number of Sound Sources
Haoran Yin, Meng Ge, Yanjie Fu +5
Recent neural network based Direction of Arrival (DoA) estimation algorithms have performed well on unknown number of sound sources scenarios. These algorithms are usually achieved…
Iterative Sound Source Localization for Unknown Number of Sources
Yanjie Fu, Meng Ge, Haoran Yin +4
Sound source localization aims to seek the direction of arrival (DOA) of all sound sources from the observed multi-channel audio. For the practical problem of unknown number of sou…
Talking Head Generation Driven by Speech-Related Facial Action Units and Audio- Based on Multimodal Representation Fusion
Sen Chen, Zhilei Liu, Jiaxing Liu +1
Talking head generation is to synthesize a lip-synchronized talking head video by inputting an arbitrary face image and corresponding audio clips. Existing methods ignore not only…