87 citations · 224 across the 60 of their papers we have counts for
22 papers · 1 filter
Lip-Focused Audio-Visual Consistency Detects Speech Content Manipulation beyond Coarse Synchronization
Jian Zong, Chenghan Lin, Meng Liu +6
Speech-content manipulation changes what a visible person appears to say by replacing the speech track and, in stronger attacks, regenerating mouth motion to match the forged utter…
AD-Mamba: Architecture-Complementary Knowledge Distillation for Efficient and Robust Audio Classification
Junyu Wang, Tianrui Wang, Meng Ge +3
Recent advancements in large-scale audio classification have been primarily driven by Transformer-based architectures due to their strong capacity in modeling long-range dependenci…
EChO-Agent: Evidence Chain Orchestration Agent for Audio Reasoning
Siyuan Zhang, Jian Zong, Junyu Wang +7
While LALMs show promise on audio question answering, they fail to focus on question-relevant segments of audio and provide a clear, checkable reasoning process when dealing with c…
Evaluating the Expressive Appropriateness of Speech in Rich Contexts
Tianrui Wang, Ziyang Ma, Yizhou Peng +26
Evaluating expressive speech remains challenging, as existing methods mainly assess emotional intensity and overlook whether a speech sample is expressively appropriate for its con…
UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions
Chunyu Qiang, Xiaopeng Wang, Kang Yin +11
Generative audio modeling has largely been fragmented into specialized tasks, text-to-speech (TTS), text-to-music (TTM), and text-to-audio (TTA), each operating under heterogeneous…
InstructAudio: Unified speech and music generation with natural language instruction
Chunyu Qiang, Kang Yin, Xiaopeng Wang +8
Text-to-speech (TTS) and text-to-music (TTM) models face significant limitations in instruction-based control. TTS systems usually depend on reference audio for timbre, offer only…
LORT: Locally Refined Convolution and Taylor Transformer for Monaural Speech Enhancement
Junyu Wang, Zizhen Lin, Tianrui Wang +3
Achieving superior enhancement performance while maintaining a low parameter count and computational complexity remains a challenge in the field of speech enhancement. In this pape…
Word-Level Emotional Expression Control in Zero-Shot Text-to-Speech Synthesis
Tianrui Wang, Haoyu Wang, Meng Ge +10
While emotional text-to-speech (TTS) has made significant progress, most existing research remains limited to utterance-level emotional expression and fails to support word-level c…
SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codec
Chunyu Qiang, Haoyu Wang, Cheng Gong +10
Speech codecs serve as a crucial bridge in unifying speech and text language models. Existing codec methods face several challenges in semantic encoding, such as residual paralingu…
Efficient Emotion and Speaker Adaptation in LLM-Based TTS via Characteristic-Specific Partial Fine-Tuning
Tianrui Wang, Meng Ge, Cheng Gong +11
While LLM-based TTS models exhibit zero-shot emotion and speaker cloning, their cloning fidelity and pronunciation clarity degrade on unseen domains. Fine-tuning is essential for a…
Progressive Residual Extraction based Pre-training for Speech Representation Learning
Tianrui Wang, Jin Li, Ziyang Ma +8
Self-supervised learning (SSL) has garnered significant attention in speech processing, excelling in linguistic tasks such as speech recognition. However, jointly improving the per…
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing
Chunyu Qiang, Wang Geng, Yi Zhao +12
Deep learning has brought significant improvements to the field of cross-modal representation learning. For tasks such as text-to-speech (TTS), voice conversion (VC), and automatic…
A Refining Underlying Information Framework for Monaural Speech Enhancement
Rui Cao, Tianrui Wang, Meng Ge +2
Supervised speech enhancement has gained significantly from recent advancements in neural networks, especially due to their ability to non-linearly fit the diverse representations…
Learning Speech Representation From Contrastive Token-Acoustic Pretraining
Chunyu Qiang, Hao Li, Yixin Tian +4
For fine-grained generation and recognition tasks such as minimally-supervised text-to-speech (TTS), voice conversion (VC), and automatic speech recognition (ASR), the intermediate…
Locate and Beamform: Two-dimensional Locating All-neural Beamformer for Multi-channel Speech Separation
Yanjie Fu, Meng Ge, Honglong Wang +7
Recently, stunning improvements on multi-channel speech separation have been achieved by neural beamformers when direction information is available. However, most of them neglect t…
MIMO-DBnet: Multi-channel Input and Multiple Outputs DOA-aware Beamforming Network for Speech Separation
Yanjie Fu, Haoran Yin, Meng Ge +5
Recently, many deep learning based beamformers have been proposed for multi-channel speech separation. Nevertheless, most of them rely on extra cues known in advance, such as speak…
Monolingual Recognizers Fusion for Code-switching Speech Recognition
Tongtong Song, Qiang Xu, Haoyu Lu +5
The bi-encoder structure has been intensively investigated in code-switching (CS) automatic speech recognition (ASR). However, most existing methods require the structures of two m…
MIMO-DoAnet: Multi-channel Input and Multiple Outputs DoA Network with Unknown Number of Sound Sources
Haoran Yin, Meng Ge, Yanjie Fu +5
Recent neural network based Direction of Arrival (DoA) estimation algorithms have performed well on unknown number of sound sources scenarios. These algorithms are usually achieved…
Iterative Sound Source Localization for Unknown Number of Sources
Yanjie Fu, Meng Ge, Haoran Yin +4
Sound source localization aims to seek the direction of arrival (DOA) of all sound sources from the observed multi-channel audio. For the practical problem of unknown number of sou…
L-SpEx: Localized Target Speaker Extraction
Meng Ge, Chenglin Xu, Longbiao Wang +3
Speaker extraction aims to extract the target speaker's voice from a multi-talker speech mixture given an auxiliary reference utterance. Recent studies show that speaker extraction…
Multi-stage Speaker Extraction with Utterance and Frame-Level Reference Signals
Meng Ge, Chenglin Xu, Longbiao Wang +3
Speaker extraction requires a sample speech from the target speaker as the reference. However, enrolling a speaker with a long speech is not practical. We propose a speaker extract…
SpEx+: A Complete Time Domain Speaker Extraction Network
Meng Ge, Chenglin Xu, Longbiao Wang +3
Speaker extraction aims to extract the target speech signal from a multi-talker environment given a target speaker's reference speech. We recently proposed a time-domain solution,…