6 papers
ReDesign: Recovering Editable Design Structures from Images via Agentic Decomposition
Jooyeol Yun, Jintae Park, Hyesu Lim +3
Recovering an editable design file from a raster image is a common and costly bottleneck in modern design workflows, yet remains challenging since editability depends on recovering…
VisualScratchpad: Inference-time Visual Concepts Analysis in Vision Language Models
Hyesu Lim, Jinho Choi, Taekyung Kim +3
High-performing vision language models still produce incorrect answers, yet their failure modes are often difficult to explain. To make model internals more accessible and enable s…
ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts
Jinho Choi, Hyesu Lim, Steffen Schneider +1
Dataset bias, where data points are skewed to certain concepts, is ubiquitous in machine learning datasets. Yet, systematically identifying these biases is challenging without cost…
CytoSAE: Interpretable Cell Embeddings for Hematology
Muhammed Furkan Dasdelen, Hyesu Lim, Michele Buck +3
Sparse autoencoders (SAEs) emerged as a promising tool for mechanistic interpretability of transformer-based foundation models. Very recently, SAEs were also adopted for the visual…
Sparse autoencoders reveal selective remapping of visual concepts during adaptation
Hyesu Lim, Jinho Choi, Jaegul Choo +1
Adapting foundation models for specific purposes has become a standard approach to build machine learning systems for downstream applications. Yet, it is an open question which mec…
Towards Calibrated Robust Fine-Tuning of Vision-Language Models
Changdae Oh, Hyesu Lim, Mijoo Kim +6
Improving out-of-distribution (OOD) generalization during in-distribution (ID) adaptation is a primary goal of robust fine-tuning of zero-shot models beyond naive fine-tuning. Howe…