9 papers
Breaking the Quality--Intelligibility Trade-off in Streaming Target Speaker Extraction via Deep-Feature-Anchored Preference Optimization
Shuhai Peng, Jinjiang Liu, Hui Lu +5
Generative streaming models for Target Speaker Extraction (TSE) commonly exhibit a quality--intelligibility trade-off, wherein naive optimization for perceptual audio quality tends…
StarTSE: Towards Streaming Target Speaker Extraction via Chunk-wise Interleaved Splicing of Autoregressive Language Model
Shuhai Peng, Hui Lu, Jinjiang Liu +8
While generative models have set new benchmarks for Target Speaker Extraction (TSE), their inherent reliance on global context precludes deployment in real-time applications. Direc…
From Inpainting to Editing: Unlocking Robust Mask-Free Visual Dubbing via Generative Bootstrapping
Xu He, Haoxian Zhang, Hejia Chen +7
Audio-driven visual dubbing aims to synchronize a video's lip movements with new speech but is fundamentally challenged by the lack of ideal training data: paired videos differing…
DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generation
Xu Guo, Fulong Ye, Qichao Sun +7
Recent advancements in foundation models have revolutionized joint audio-video generation. However, existing approaches typically treat human-centric tasks including reference-base…
StableDub: Taming Diffusion Prior for Generalized and Efficient Visual Dubbing
Liyang Chen, Tianze Zhou, Xu He +7
The visual dubbing task aims to generate mouth movements synchronized with the driving audio, which has seen significant progress in recent years. However, two critical deficiencie…
HuMo: Human-Centric Video Generation via Collaborative Multi-Modal Conditioning
Liyang Chen, Tianxiang Ma, Jiawei Liu +7
Human-Centric Video Generation (HCVG) methods seek to synthesize human videos from multimodal inputs, including text, image, and audio. Existing methods struggle to effectively coo…