12 papers
Benchmarking Visual State Tracking in Multimodal Video Understanding
Sihyun Yu, Nanye Ma, Pinzhi Huang +8
Understanding a video requires more than recognizing isolated moments, as humans continuously track entities, states, and events over time. This capacity for visual state tracking…
Improving Motion in Image-to-Video Models via Adaptive Low-Pass Guidance
June Suk Choi, Kyungmin Lee, Sihyun Yu +3
Recent text-to-video (T2V) models have demonstrated strong capabilities in producing high-quality, dynamic videos. To improve the visual controllability, recent works have consider…
World Action Models are Zero-shot Policies
Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng +33
State-of-the-art Vision-Language-Action (VLA) models excel at semantic generalization but struggle to generalize to unseen physical motions in novel environments. We introduce Drea…
AtomMOF: All-Atom Flow Matching for MOF-Adsorbate Structure Prediction
Nayoung Kim, Honghui Kim, Sihyun Yu +3
Deep generative models have shown promise for modeling metal-organic frameworks (MOFs), but existing approaches (1) rely on coarse-grained representations that assume fixed bond le…
DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos
Shenyuan Gao, William Liang, Kaiyuan Zheng +27
Being able to simulate the outcomes of actions in varied environments will revolutionize the development of generalist agents at scale. However, modeling these world dynamics, espe…
Decoupled MeanFlow: Turning Flow Models into Flow Maps for Accelerated Sampling
Kyungmin Lee, Sihyun Yu, Jinwoo Shin
Denoising generative models, such as diffusion and flow-based models, produce high-quality samples but require many denoising steps due to discretization error. Flow maps, which es…