27 citations · 58 across the 22 of their papers we have counts for
22 papers
Benchmarking Visual State Tracking in Multimodal Video Understanding
Sihyun Yu, Nanye Ma, Pinzhi Huang +8
Understanding a video requires more than recognizing isolated moments, as humans continuously track entities, states, and events over time. This capacity for visual state tracking…
World Action Models are Zero-shot Policies
Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng +33
State-of-the-art Vision-Language-Action (VLA) models excel at semantic generalization but struggle to generalize to unseen physical motions in novel environments. We introduce Drea…
AtomMOF: All-Atom Flow Matching for MOF-Adsorbate Structure Prediction
Nayoung Kim, Honghui Kim, Sihyun Yu +3
Deep generative models have shown promise for modeling metal-organic frameworks (MOFs), but existing approaches (1) rely on coarse-grained representations that assume fixed bond le…
DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos
Shenyuan Gao, William Liang, Kaiyuan Zheng +27
Being able to simulate the outcomes of actions in varied environments will revolutionize the development of generalist agents at scale. However, modeling these world dynamics, espe…
Decoupled MeanFlow: Turning Flow Models into Flow Maps for Accelerated Sampling
Kyungmin Lee, Sihyun Yu, Jinwoo Shin
Denoising generative models, such as diffusion and flow-based models, produce high-quality samples but require many denoising steps due to discretization error. Flow maps, which es…
ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Context
Huiwon Jang, Sihyun Yu, Heeseung Kwon +3
Leveraging temporal context is crucial for success in partially observable robotic tasks. However, prior work in behavior cloning has demonstrated inconsistent performance gains wh…
Improving Motion in Image-to-Video Models via Adaptive Low-Pass Guidance
June Suk Choi, Kyungmin Lee, Sihyun Yu +3
Recent text-to-video (T2V) models have demonstrated strong capabilities in producing high-quality, dynamic videos. To improve the visual controllability, recent works have consider…
Learning Flexible Forward Trajectories for Masked Molecular Diffusion
Hyunjin Seo, Taewon Kim, Sihyun Yu +1
Masked diffusion models (MDMs) have achieved notable progress in modeling discrete data, while their potential in molecular generation remains underexplored. In this work, we explo…
MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation
Sihyun Yu, Meera Hahn, Dan Kondratyuk +6
Diffusion models are successful for synthesizing high-quality videos but are limited to generating short clips (e.g., 2-10 seconds). Synthesizing sustained footage (e.g. over minut…
Controllable Human Image Generation with Personalized Multi-Garments
Yisol Choi, Sangkyung Kwak, Sihyun Yu +2
We present BootComp, a novel framework based on text-to-image diffusion models for controllable human image generation with multiple reference garments. Here, the main bottleneck i…
Efficient Long Video Tokenization via Coordinate-based Patch Reconstruction
Huiwon Jang, Sihyun Yu, Jinwoo Shin +2
Efficient tokenization of videos remains a challenge in training vision models that can process long videos. One promising direction is to develop a tokenizer that can encode long…
Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think
Sihyun Yu, Sangkyung Kwak, Huiwon Jang +4
Recent studies have shown that the denoising process in (generative) diffusion models can induce meaningful (discriminative) representations inside the model, though the quality of…
Data-Efficient Molecular Generation with Hierarchical Textual Inversion
Seojin Kim, Jaehyun Nam, Sihyun Yu +2
Developing an effective molecular generation framework even with a limited number of molecules is often important for its practical deployment, e.g., drug discovery, since acquirin…
Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition
Sihyun Yu, Weili Nie, De-An Huang +3
Video diffusion models have recently made great progress in generation quality, but are still limited by the high memory and computational requirements. This is because current vid…
Video Probabilistic Diffusion Models in Projected Latent Space
Sihyun Yu, Kihyuk Sohn, Subin Kim +1
Despite the remarkable progress in deep generative models, synthesizing high-resolution and temporally coherent videos still remains a challenge due to their high-dimensionality an…
Enhancing Multiple Reliability Measures via Nuisance-extended Information Bottleneck
Jongheon Jeong, Sihyun Yu, Hankook Lee +1
In practical scenarios where training data is limited, many predictive signals in the data can be rather from some biases in data acquisition (i.e., less generalizable), so that on…
Learning Large-scale Neural Fields via Context Pruned Meta-Learning
Jihoon Tack, Subin Kim, Sihyun Yu +3
We introduce an efficient optimization-based meta-learning technique for large-scale neural field training by realizing significant memory savings through automated online context…
Scalable Neural Video Representations with Learnable Positional Features
Subin Kim, Sihyun Yu, Jaeho Lee +1
Succinct representation of complex signals using coordinate-based neural representations (CNRs) has seen great progress, and several recent efforts focus on extending them for hand…
Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks
Sihyun Yu, Jihoon Tack, Sangwoo Mo +4
In the deep learning era, long video generation of high-quality still remains challenging due to the spatio-temporal complexity and continuity of videos. Existing prior works have…
RoMA: Robust Model Adaptation for Offline Model-based Optimization
Sihyun Yu, Sungsoo Ahn, Le Song +1
We consider the problem of searching an input maximizing a black-box objective function given a static dataset of input-output queries. A popular approach to solving this problem i…
Abstract Reasoning via Logic-guided Generation
Sihyun Yu, Sangwoo Mo, Sungsoo Ahn +1
Abstract reasoning, i.e., inferring complicated patterns from given observations, is a central building block of artificial general intelligence. While humans find the answer by ei…
Consistency Regularization for Adversarial Robustness
Jihoon Tack, Sihyun Yu, Jongheon Jeong +3
Adversarial training (AT) is currently one of the most successful methods to obtain the adversarial robustness of deep neural networks. However, the phenomenon of robust overfittin…