1 citations · 2 across the 18 of their papers we have counts for
18 papers
When Text Misleads: Inconsistent-Aware Reasoning for Audio-Grounded Dialogue
Yen-Ju Lu, Yuzhe Wang, Yaohan Guan +8
Understanding spoken dialogue requires joint reasoning over lexical content and paralinguistic acoustic signals such as emotion and conversational intent. However, existing evaluat…
VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentation
Jiarui Hai, Karan Thakkar, Ke Chen +5
Recent breakthroughs in generative models have made text-to-voice generation (TTV) possible, enabling the synthesis of speech directly from textual voice descriptions. However, exi…
ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models
Fengjie Lu, Chenang Jiang, Jiarui Hai +2
Recent advances in language--audio retrieval have been largely driven by contrastive dual-encoder architectures that align audio and text in a shared embedding space. While effecti…
AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
Xilin Jiang, Qiaolin Wang, Junkai Wu +30
Internet audio-visual clips convey meaning through time-varying sound and motion, which extend beyond what text alone can represent. To examine whether AI models can understand suc…
Summary of The Inaugural Music Source Restoration Challenge
Yongyi Zang, Jiarui Hai, Wanying Ge +5
Music Source Restoration (MSR) aims to recover original, unprocessed instrument stems from professionally mixed and degraded audio, requiring the reversal of both production effect…
Adapting Speech Language Model to Singing Voice Synthesis
Yiwen Zhao, Jiatong Shi, Jinchuan Tian +4
Speech Language Models (SLMs) have recently emerged as a unified paradigm for addressing a wide range of speech-related tasks, including text-to-speech (TTS), speech enhancement (S…
MSRBench: A Benchmarking Dataset for Music Source Restoration
Yongyi Zang, Jiarui Hai, Wanying Ge +5
Music Source Restoration (MSR) extends source separation to realistic settings where signals undergo production effects (equalization, compression, reverb) and real-world degradati…
FlexSED: Towards Open-Vocabulary Sound Event Detection
Jiarui Hai, Helin Wang, Weizhe Guo +1
Despite recent progress in large-scale sound event detection (SED) systems capable of handling hundreds of sound classes, existing multi-class classification frameworks remain fund…
SynSonic: Augmenting Sound Event Detection through Text-to-Audio Diffusion ControlNet and Effective Sample Filtering
Jiarui Hai, Mounya Elhilali
Data synthesis and augmentation are essential for Sound Event Detection (SED) due to the scarcity of temporally labeled data. While augmentation methods like SpecAugment and Mix-up…
CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speech
Helin Wang, Jiarui Hai, Dading Chong +11
Recent advancements in generative artificial intelligence have significantly transformed the field of style-captioned text-to-speech synthesis (CapTTS). However, adapting CapTTS to…
SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipeline
Helin Wang, Jiarui Hai, Dongchao Yang +7
Target Speech Extraction (TSE) aims to isolate a target speaker's voice from a mixture of multiple speakers by leveraging speaker-specific cues, typically provided as auxiliary aud…
EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer
Jiarui Hai, Yong Xu, Hao Zhang +4
We introduce EzAudio, a text-to-audio (T2A) generation framework designed to produce high-quality, natural-sounding sound effects. Core designs include: (1) We propose EzAudio-DiT,…
SoloAudio: Target Sound Extraction with Language-oriented Audio Diffusion Transformer
Helin Wang, Jiarui Hai, Yen-Ju Lu +3
In this paper, we introduce SoloAudio, a novel diffusion-based generative model for target sound extraction (TSE). Our approach trains latent diffusion models on audio, replacing t…
SSR-Speech: Towards Stable, Safe and Robust Zero-shot Text-based Speech Editing and Synthesis
Helin Wang, Meng Yu, Jiarui Hai +5
In this paper, we introduce SSR-Speech, a neural codec autoregressive model designed for stable, safe, and robust zero-shot textbased speech editing and text-to-speech synthesis. S…
DreamVoice: Text-Guided Voice Conversion
Jiarui Hai, Karan Thakkar, Helin Wang +2
Generative voice technologies are rapidly evolving, offering opportunities for more personalized and inclusive experiences. Traditional one-shot voice conversion (VC) requires a ta…
Noise-robust Speech Separation with Fast Generative Correction
Helin Wang, Jesus Villalba, Laureano Moro-Velazquez +3
Speech separation, the task of isolating multiple speech sources from a mixed audio signal, remains challenging in noisy environments. In this paper, we propose a generative correc…
Investigating Self-Supervised Deep Representations for EEG-based Auditory Attention Decoding
Karan Thakkar, Jiarui Hai, Mounya Elhilali
Auditory Attention Decoding (AAD) algorithms play a crucial role in isolating desired sound sources within challenging acoustic environments directly from brain activity. Although…
DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extraction
Jiarui Hai, Helin Wang, Dongchao Yang +3
Common target sound extraction (TSE) approaches primarily relied on discriminative approaches in order to separate the target sound while minimizing interference from the unwanted…