9 papers
DynaVieW: Schema-Guided World Modeling for Understanding Hierarchical Visual Dynamics
Silin Gao, Hao Zhao, Zeming Chen +8
Multimodal LLMs struggle to systematically model the temporal evolution of visual scenes in videos or multi-image sequences. Such inputs require models to predict or simulate multi…
MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation
JoungBin Lee, Jaewoo Jung, Jongmin Lee +8
Synthesizing a novel-view video from a monocular reference video along a target camera trajectory requires both geometric consistency and motion fidelity with respect to the refere…
4D Reconstruction from Sparse Dynamic Cameras
Kazuki Ozeki, Shun Kenney, Yuto Shibata +6
Although dynamic 3D (i.e., 4D) reconstruction from a monocular dynamic camera has recently advanced, it remains fundamentally limited by depth ambiguity. In this paper, we focus on…
Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction
Mungyeom Kim, Minkyeong Jeon, Honggyu An +10
Dynamic scene reconstruction from monocular video remains a fundamental challenge in computer vision. Existing feed-forward methods predict 3D Gaussians pixel-wise for each frame,…
Echoes Over Time: Unlocking Length Generalization in Video-to-Audio Generation Models
Christian Simon, Masato Ishii, Wei-Yao Wang +8
Scaling multimodal alignment between video and audio is challenging, particularly due to limited data and the mismatch between text descriptions and frame-level video information.…
Coherent Audio-Visual Editing via Conditional Audio Generation Following Video Edits
Masato Ishii, Akio Hayakawa, Takashi Shibuya +1
We introduce a novel pipeline for joint audio-visual editing that enhances the coherence between edited video and its accompanying audio. Our approach first applies state-of-the-ar…