7 papers
Vision-driven Preference Synthesis for Mitigating Hallucinations in VLMs
Yunhun Nam, Jongheon Jeong
Vision-Language Models (VLMs) have shown strong performance in visual understanding, yet they still suffer from hallucinations, generating content that is not grounded in the image…
Persona-Pruner: Sculpting Lightweight Models for Role-Playing
Jinsu Kim, Jihoon Tack, Noah Lee +1
Language Models (LMs) have shown remarkable potential as role-playing chatbots, delivering consistent, stylized interactions when given a specification of a character or user perso…
Margin-aware Preference Optimization for Aligning Diffusion Models without Reference
Jiwoo Hong, Sayak Paul, Noah Lee +3
Modern preference alignment methods, such as DPO, rely on divergence regularization to a reference model for training stability-but this creates a fundamental problem we call "refe…
Learning from the Undesirable: Robust Adaptation of Language Models without Forgetting
Yunhun Nam, Jaehyung Kim, Jongheon Jeong
Language models (LMs) are often adapted through supervised fine-tuning (SFT) to specialize their capabilities for downstream tasks. However, in typical scenarios where the fine-tun…
BlurGuard: A Simple Approach for Robustifying Image Protection Against AI-Powered Editing
Jinsu Kim, Yunhun Nam, Minseon Kim +2
Recent advances in text-to-image models have increased the exposure of powerful image editing techniques as a tool, raising concerns about their potential for malicious use. An eme…
EXPERT: An Explainable Image Captioning Evaluation Metric with Structured Explanations
Hyunjong Kim, Sangyeop Kim, Jongheon Jeong +2
Recent advances in large language models and vision-language models have led to growing interest in explainable evaluation metrics for image captioning. However, these metrics gene…