5 papers
TV-AudioRemover: Joint Text-Visual Guided Sound Removal with Multi-Task Hard-Mixture Curriculum
Xinyue Guo, Jianxuan Yang, Daiguo Zhou +4
Visual object removal can eliminate a target from video frames, yet its acoustic trace persists in the soundtrack, causing obvious audio-visual inconsistency. Existing video inpain…
ControlFoley: Unified and Controllable Video-to-Audio Generation with Cross-Modal Conflict Handling
Jianxuan Yang, Xinyue Guo, Zhi Cheng +10
Recent advances in video-to-audio (V2A) generation enable high-quality audio synthesis from visual content, yet achieving robust and fine-grained controllability remains challengin…
AV-Edit: Multimodal Generative Sound Effect Editing via Audio-Visual Semantic Joint Control
Xinyue Guo, Xiaoran Yang, Lipan Zhang +3
Sound effect editing-modifying audio by adding, removing, or replacing elements-remains constrained by existing approaches that rely solely on low-level signal processing or coarse…
MeanFlow-Accelerated Multimodal Video-to-Audio Synthesis via One-Step Generation
Xiaoran Yang, Jianxuan Yang, Xinyue Guo +3
A key challenge in synthesizing audios from silent videos is the inherent trade-off between synthesis quality and inference efficiency in existing methods. For instance, flow match…
MultiSoundGen: Video-to-Audio Generation for Multi-Event Scenarios via SlowFast Contrastive Audio-Visual Pretraining and Direct Preference Optimization
Jianxuan Yang, Xiaoran Yang, Lipan Zhang +3
Current video-to-audio (V2A) methods struggle in complex multi-event scenarios (video scenarios involving multiple sound sources, sound events, or transitions) due to two critical…