11 papers
Reimagine Video Dynamics
Yu Yuan, Yawen Lu, Guoxian Song +5
Most video editing methods focus on changing the appearance of the source video, while offering limited control over its dynamics. We introduce Reimagine Video Dynamics (RVD), a fr…
PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignment
Zhexiao Xiong, Yizhi Song, Liu He +4
Video Diffusion Models (VDMs) offer a promising approach for simulating dynamic scenes and environments, with broad applications in robotics and media generation. However, existing…
SeeU: Seeing the Unseen World via 4D Dynamics-aware Generation
Yu Yuan, Tharindu Wickremasinghe, Zeeshan Nadir +3
Images and videos are discrete 2D projections of the 4D world (3D space + time). Most visual understanding, prediction, and generation operate directly on 2D observations, leading…
Progressive Image Restoration via Text-Conditioned Video Generation
Peng Kang, Xijun Wang, Yu Yuan
Recent text-to-video models have demonstrated strong temporal generation capabilities, yet their potential for image restoration remains underexplored. In this work, we repurpose C…
NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamics
Yu Yuan, Xijun Wang, Tharindu Wickremasinghe +3
A primary bottleneck in large-scale text-to-video generation today is physical consistency and controllability. Despite recent advances, state-of-the-art models often produce unrea…
Astrophotography turbulence mitigation via generative models
Joonyeoup Kim, Yu Yuan, Xingguang Zhang +2
Photography is the cornerstone of modern astronomical and space research. However, most astronomical images captured by ground-based telescopes suffer from atmospheric turbulence,…
iHDR: Iterative HDR Imaging with Arbitrary Number of Exposures
Yu Yuan, Yiheng Chi, Xingguang Zhang +1
High dynamic range (HDR) imaging aims to obtain a high-quality HDR image by fusing information from multiple low dynamic range (LDR) images. Numerous learning-based HDR imaging met…
Learning Phase Distortion with Selective State Space Models for Video Turbulence Mitigation
Xingguang Zhang, Nicholas Chimitt, Xijun Wang +2
Atmospheric turbulence is a major source of image degradation in long-range imaging systems. Although numerous deep learning-based turbulence mitigation (TM) methods have been prop…
Personalized Generative Low-light Image Denoising and Enhancement
Xijun Wang, Prateek Chennuri, Dilshan Godaliyadda +5
Modern cameras' performance in low-light conditions remains suboptimal due to fundamental limitations in photon shot noise and sensor read noise. Generative image restoration metho…
Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesis
Yu Yuan, Xijun Wang, Yichen Sheng +3
Image generation today can produce somewhat realistic images from text prompts. However, if one asks the generator to synthesize a specific camera setting such as creating differen…
Improving Decoupled Posterior Sampling for Inverse Problems using Data Consistency Constraint
Zhi Qi, Shihong Yuan, Yulin Yuan +3
Diffusion models have shown strong performances in solving inverse problems through posterior sampling while they suffer from errors during earlier steps. To mitigate this issue, s…