Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy

Abhinav Sharma, Sai Karthik Navuluru, Wang Wei +21

Video and audio are perceived together, yet most generative models treat them in isolation. We examine methods that model the two modalities jointly, generate one from the other, o…