9 papers
CoSyncDiT: Cognitive Synchronous Diffusion Transformer for Movie Dubbing
Gaoxiang Cong, Liang Li, Jiaxin Ye +4
Movie dubbing aims to synthesize speech that preserves the vocal identity of a reference audio while synchronizing with the lip movements in a target video. Existing methods fail t…
Question-guided Visual Compression with Memory Feedback for Long-Term Video Understanding
Sosuke Yamao, Natsuki Miyahara, Yuankai Qi +1
In the context of long-term video understanding with large multimodal models, many frameworks have been proposed. Although transformer-based visual compressors and memory-augmented…
InstructDubber: Instruction-based Alignment for Zero-shot Movie Dubbing
Zhedong Zhang, Liang Li, Gaoxiang Cong +5
Movie dubbing seeks to synthesize speech from a given script using a specific voice, while ensuring accurate lip synchronization and emotion-prosody alignment with the character's…
FlowDubber: Movie Dubbing with LLM-based Semantic-aware Learning and Flow Matching based Voice Enhancing
Gaoxiang Cong, Liang Li, Jiadong Pan +5
Movie Dubbing aims to convert scripts into speeches that align with the given movie clip in both temporal and emotional aspects while preserving the vocal timbre of a given brief r…
EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbing
Gaoxiang Cong, Jiadong Pan, Liang Li +5
Given a piece of text, a video clip, and a reference audio, the movie dubbing task aims to generate speech that aligns with the video while cloning the desired voice. The existing…
ProgRoCC: A Progressive Approach to Rough Crowd Counting
Shengqin Jiang, Linfei Li, Haokui Zhang +6
As the number of individuals in a crowd grows, enumeration-based techniques become increasingly infeasible and their estimates increasingly unreliable. We propose instead an estima…