5 citations · 6 across the 10 of their papers we have counts for
12 papers
Towards Expressive and Faithful Audio-to-Image Generation: A Unified Multimodal Dataset and Synthesis Framework
Dongxu Ge, Shansong Liu, Cheng Gong +3
As an important subfield of cross-modal generation, synthesizing static visual content in the form of images from audio, namely audio-to-image (A2I) generation, has attracted incre…
High-Fidelity Generative Audio Compression at 0.275kbps
Hao Ma, Ruihao Jing, Shansong Liu +4
High-fidelity general audio compression at ultra-low bitrates is crucial for applications ranging from low-bandwidth communication to generative audio-language modeling. Traditiona…
Rare Word Recognition and Translation Without Fine-Tuning via Task Vector in Speech Models
Ruihao Jing, Cheng Gong, Yu Jiang +5
Rare words remain a critical bottleneck for speech-to-text systems. While direct fine-tuning improves recognition of target words, it often incurs high cost, catastrophic forgettin…
Towards Multimodal Query-Based Spatial Audio Source Extraction
Chenxin Yu, Hao Ma, Xu Li +4
Query-based audio source extraction seeks to recover a target source from a mixture conditioned on a query. Existing approaches are largely confined to single-channel audio, leavin…
FoleySpace: Vision-Aligned Binaural Spatial Audio Generation
Lei Zhao, Rujin Chen, Chi Zhang +2
Recently, with the advancement of AIGC, deep learning-based video-to-audio (V2A) technology has garnered significant attention. However, existing research mostly focuses on mono au…
HAMLET: A Hierarchical and Adaptive Multi-Agent Framework for Live Embodied Theatrics
Shufan Jiang, Sizhou Chen, Chios Chen +3
Creating an immersive and interactive theatrical experience is a long-term goal in the field of interactive narrative. The emergence of large language models (LLMs) provides a new…
AudioSpa: Spatializing Sound Events with Text
Linfeng Feng, Lei Zhao, Boyu Zhu +2
Text-to-audio (TTA) systems have recently demonstrated strong performance in synthesizing monaural audio from text. However, the task of generating binaural spatial audio from text…
DualSpec: Text-to-spatial-audio Generation via Dual-Spectrogram Guided Diffusion Model
Lei Zhao, Sizhou Chen, Linfeng Feng +4
Text-to-audio (TTA), which generates audio signals from textual descriptions, has received huge attention in recent years. However, recent works focused on text to monaural audio o…
UniForm: A Unified Multi-Task Diffusion Transformer for Audio-Video Generation
Lei Zhao, Linfeng Feng, Dongxu Ge +5
With the rise of diffusion models, audio-video generation has been revolutionized. However, most existing methods rely on separate modules for each modality, with limited explorati…
Enhancing Intelligibility for Generative Target Speech Extraction via Joint Optimization with Target Speaker ASR
Hao Ma, Rujin Chen, Xiao-Lei Zhang +2
Target speech extraction (TSE) isolates the speech of a specific speaker from a multi-talker overlapped speech mixture. Most existing TSE models rely on discriminative methods, typ…
Eliminating Quantization Errors in Classification-Based Sound Source Localization
Linfeng Feng, Xiao-Lei Zhang, Xuelong Li
Sound Source Localization (SSL) involves estimating the Direction of Arrival (DOA) of sound sources. Since the DOA estimation output space is continuous, regression might be more s…
Diffusion-Based Adversarial Purification for Speaker Verification
Yibo Bai, Xiao-Lei Zhang, Xuelong Li
Recently, automatic speaker verification (ASV) based on deep learning is easily contaminated by adversarial attacks, which is a new type of attack that injects imperceptible pertur…