4 papers
Interpreting Attention Heads for Image-to-Text Information Flow in Large Vision-Language Models
Jinyeong Kim, Seil Kang, Jiwoo Park +2
Large Vision-Language Models (LVLMs) answer visual questions by transferring information from images to text through a series of attention heads. While this image-to-text informati…
Mono-Modalizing Extremely Heterogeneous Multi-Modal Medical Image Registration
Kyobin Choo, Hyunkyung Han, Jinyeong Kim +2
In clinical practice, imaging modalities with functional characteristics, such as positron emission tomography (PET) and fractional anisotropy (FA), are often aligned with a struct…
See What You Are Told: Visual Attention Sink in Large Multimodal Models
Seil Kang, Jinyeong Kim, Junhyeok Kim +1
Large multimodal models (LMMs) "see" images by leveraging the attention mechanism between text and visual tokens in the transformer decoder. Ideally, these models should focus on k…
Pathology-Aware Adaptive Watermarking for Text-Driven Medical Image Synthesis
Chanyoung Kim, Dayun Ju, Jinyeong Kim +3
As recent text-conditioned diffusion models have enabled the generation of high-quality images, concerns over their potential misuse have also grown. This issue is critical in the…