InstructDubber: Instruction-based Alignment for Zero-shot Movie Dubbing

Zhedong Zhang, Liang Li, Gaoxiang Cong +5

Movie dubbing seeks to synthesize speech from a given script using a specific voice, while ensuring accurate lip synchronization and emotion-prosody alignment with the character's…