12 papers
Rare Text Semantics Were Always There in Your Diffusion Transformer
Seil Kang, Woojung Han, Dayun Ju +1
Starting from flow- and diffusion-based transformers, Multi-modal Diffusion Transformers (MM-DiTs) have reshaped text-to-vision generation, gaining acclaim for exceptional visual f…
Interpreting Attention Heads for Image-to-Text Information Flow in Large Vision-Language Models
Jinyeong Kim, Seil Kang, Jiwoo Park +2
Large Vision-Language Models (LVLMs) answer visual questions by transferring information from images to text through a series of attention heads. While this image-to-text informati…
Interpreting vision transformers via residual replacement model
Jinyeong Kim, Junhyeok Kim, Yumin Shim +3
How do vision transformers (ViTs) represent and process the world? This paper addresses this long-standing question through the first systematic analysis of 6.6K features across al…
Mono-Modalizing Extremely Heterogeneous Multi-Modal Medical Image Registration
Kyobin Choo, Hyunkyung Han, Jinyeong Kim +2
In clinical practice, imaging modalities with functional characteristics, such as positron emission tomography (PET) and fractional anisotropy (FA), are often aligned with a struct…
Backbone Augmented Training for Adaptations
Jae Wan Park, Junhyeok Kim, Youngjun Jun +2
Adaptations facilitate efficient training of large backbone models, including diffusion models for image generation and transformer-based language models. While various adaptation…
WAVE: Warp-Based View Guidance for Consistent Novel View Synthesis Using a Single Image
Jiwoo Park, Tae Eun Choi, Youngjun Jun +1
Generating high-quality novel views of a scene from a single image requires maintaining structural coherence across different views, referred to as view consistency. While diffusio…