6 papers
Token Warping Helps MLLMs Look from Nearby Viewpoints
Phillip Y. Lee, Chanho Park, Mingue Park +3
Can warping tokens, rather than pixels, help multimodal large language models (MLLMs) understand how a scene appears from a nearby viewpoint? While MLLMs perform well on visual rea…
Unconditional Priors Matter! Improving Conditional Generation of Fine-Tuned Diffusion Models
Prin Phunyaphibarn, Phillip Y. Lee, Jaihoon Kim +1
Classifier-Free Guidance (CFG) is a fundamental technique in training conditional diffusion models. The common practice for CFG-based training is to use a single network to learn b…
Toward Ambulatory Vision: Learning Visually-Grounded Active View Selection
Juil Koo, Daehyeon Choi, Sangwoo Youn +2
Vision Language Models (VLMs) excel at visual question answering (VQA) but remain limited to snapshot vision, reasoning from static images. In contrast, embodied agents require amb…
DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models
Mingue Park, Prin Phunyaphibarn, Phillip Y. Lee +1
We introduce DiverseVAR, a framework that enhances the diversity of text-conditioned visual autoregressive models (VAR) at test time without requiring retraining, fine-tuning, or s…
Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation
Phillip Y. Lee, Jihyeon Je, Chanho Park +3
We present a framework for perspective-aware reasoning in vision-language models (VLMs) through mental imagery simulation. Perspective-taking, the ability to perceive an environmen…
GrounDiT: Grounding Diffusion Transformers via Noisy Patch Transplantation
Phillip Y. Lee, Taehoon Yoon, Minhyuk Sung
We introduce GrounDiT, a novel training-free spatial grounding technique for text-to-image generation using Diffusion Transformers (DiT). Spatial grounding with bounding boxes has…