5 papers
Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding
Jaehyun Jeon, Min Soo Kim, Jang Han Yoon +5
User interface (UI) design goes beyond visuals to shape user experience (UX), underscoring the shift toward UI/UX as a unified concept. While recent studies have explored UI evalua…
A11YN: aligning LLMs for accessible web UI code generation
Janghan Yoon, Jaegwan Cho, Junhyeok Kim +3
Large language models (LLMs) have recently demonstrated strong capabilities in generating functional and aesthetic web interfaces directly from instructions. However, these models…
Zero-shot Multimodal Document Retrieval via Cross-modal Question Generation
Yejin Choi, Jaewoo Park, Janghan Yoon +3
Rapid advances in Multimodal Large Language Models (MLLMs) have expanded information retrieval beyond purely textual inputs, enabling retrieval from complex real world documents th…
RA-SGG: Retrieval-Augmented Scene Graph Generation Framework via Multi-Prototype Learning
Kanghoon Yoon, Kibum Kim, Jaehyung Jeon +3
Scene Graph Generation (SGG) research has suffered from two fundamental challenges: the long-tailed predicate distribution and semantic ambiguity between predicates. These challeng…
Can visual language models resolve textual ambiguity with visual cues? Let visual puns tell you!
Jiwan Chung, Seungwon Lim, Jaehyun Jeon +2
Humans possess multimodal literacy, allowing them to actively integrate information from various modalities to form reasoning. Faced with challenges like lexical ambiguity in text,…