6 papers · 1 filter
A Composable Evaluation System for Reproducible Omni-Modal Foundation Model Evaluation
Hodong Lee, Sanghee Park, Dohoon Ryu +4
Building an omni-modal foundation model means evaluating it across text, image, video, and audio. Excellent evaluation toolkits exist for each modality, but their inference engines…
Text Capability Loss in Vision-Language Adaptation: An Attention-Sink Diagnosis
Minsik Choi, Geewook Kim, Young Geun Kim
Fine-tuning a pretrained LLM into a vision-language model (VLM) can erode the backbone's text capability, with the damage concentrated on tasks that require following exact output…
Compliance, Capability, and Conflict: Benchmarking Multimodal LLMs under System Messages
Juan Yeo, Geewook Kim
Production deployments of Multimodal Large Language Models (MLLMs) increasingly rely on system messages to govern model behavior. Yet existing benchmarks either evaluate constraint…
KCSAT-ML: Probing Reasoning Models with Nationwide-Cohort Human Difficulty
Sanghee Park, Geewook Kim, Kee-Eung Kim
Math reasoning benchmarks have proliferated, yet most lack a per-item difficulty signal grounded in actual human performance. We introduce KCSAT-ML, a decade (2014-2025) of Korean…
K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts
Nahyun Lee, Dongkeun Yoon, Guijin Son +12
Frontier model evaluations are shifting from foundational capabilities (e.g., instruction following and reasoning) toward compositional, agentic ones, but Korean agentic benchmarks…
Decentralized Instruction Tuning: Conflict-Aware Splitting and Weight Merging
Minsik Choi, Geewook Kim
Instruction tuning aligns large language models, including multimodal ones, with diverse user intents, but scaling to heterogeneous mixtures is hindered by gradient interference an…