1 citations · 1 across the 12 of their papers we have counts for
14 papers
Multimodal Safety Evaluation Should Measure Controllability Beyond Classification
Junhyeong Park, Hanwool Lee, DongGeon Lee +4
VLM safety is commonly evaluated through input- and output-level classification. Such classification is necessary, but it does not reveal whether a safety state is accessible or co…
NOLLI: A Difficulty-Calibrated Puzzle Benchmark for Diagnosing the English-Korean Performance Gap
Dasol Choi, Joonyong Park, Daegon Yu +3
We introduce NOLLI, a procedurally generated English-Korean puzzle benchmark designed to diagnose where Korean performance gaps arise. It comprises 15 puzzle types (25 tasks; 7,500…
NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms
Hanwool Lee, Dasol Choi, Bokyeong Kim +2
Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressu…
EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards
Siddhant Panpatil, Arth Singh, Mijin Koo +3
Vision-language models (VLMs) are increasingly proposed as runtime safety guards for embodied agents in homes and factories. A deployable guard must catch genuinely unsafe situatio…
COMPASS: A Framework for Evaluating Organization-Specific Policy Alignment in LLMs
Dasol Choi, DongGeon Lee, Brigitta Jesica Kartono +6
As large language models are deployed in high-stakes enterprise applications, from healthcare to finance, ensuring adherence to organization-specific policies has become essential.…
What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models
Dasol Choi, Guijin Son, Hanwool Lee +7
Current vision-language benchmarks predominantly feature well-structured questions with clear, explicit prompts. However, real user queries are often informal and underspecified. U…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought
Guijin Son, Donghun Yang, Hitesh Laxmichand Patel +9
Recent frontier models employ long chain-of-thought reasoning to explore solution spaces in context and achieve stonger performance. While many works study distillation to build sm…
Ko-PIQA: A Korean Physical Commonsense Reasoning Dataset with Cultural Context
Dasol Choi, Jungwhan Kim, Guijin Son
Physical commonsense reasoning datasets like PIQA are predominantly English-centric and lack cultural diversity. We introduce Ko-PIQA, a Korean physical commonsense reasoning datas…
When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with Benign Inputs
Hiskias Dingeto, Taeyoun Kwon, Dasol Choi +5
As large language models (LLMs) become increasingly integrated into daily life, audio has emerged as a key interface for human-AI interaction. However, this convenience also introd…
Redefining Evaluation Standards: A Unified Framework for Evaluating the Korean Capabilities of Language Models
Hanwool Lee, Dasol Choi, Sooyong Kim +6
Recent advancements in Korean large language models (LLMs) have driven numerous benchmarks and evaluation methods, yet inconsistent protocols cause up to 10 p.p performance gaps ac…
Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gap
Hyunwoo Ko, Guijin Son, Dasol Choi
Large language models (LLMs) demonstrate exceptional performance on complex reasoning tasks. However, despite their strong reasoning capabilities in high-resource languages (e.g.,…
Multi-Step Reasoning in Korean and the Emergent Mirage
Guijin Son, Hyunwoo Ko, Dasol Choi
We introduce HRMCR (HAE-RAE Multi-Step Commonsense Reasoning), a benchmark designed to evaluate large language models' ability to perform multi-step reasoning in culturally specifi…
Improving Fine-grained Visual Understanding in VLMs through Text-Only Training
Dasol Choi, Guijin Son, Soo Yong Kim +2
Visual-Language Models (VLMs) have become a powerful tool for bridging the gap between visual and linguistic understanding. However, the conventional learning approaches for VLMs o…