1 citations · 2 across the 18 of their papers we have counts for
12 papers · 1 filter
Language Shapes Instruction Hierarchy Compliance in Multilingual LLMs
Jiwon Moon, Yerin Hwang, Kyomin Jung
Instruction hierarchy (IH) requires models to prioritize instructions by source, ensuring that higher-priority instructions override lower-priority ones. Despite its importance for…
Casual as an Anchor: Resolving Supervision Misalignment in Formality Transfer Dataset
Hyojeong Yu, Hyukhun Koh, Minsung Kim +1
Formality transfer is commonly framed as a symmetric bidirectional task between informal and formal registers. We argue that this framing conceals a supervision design flaw in exis…
When Wording Steers the Evaluation: Framing Bias in LLM judges
Yerin Hwang, Dongryeol Lee, Taegwan Kang +2
Large language models (LLMs) are known to produce varying responses depending on prompt phrasing, indicating that subtle guidance in phrasing can steer their answers. However, the…
Judging Against the Reference: Uncovering Knowledge-Driven Failures in LLM-Judges on QA Evaluation
Dongryeol Lee, Yerin Hwang, Taegwan Kang +3
While large language models (LLMs) are increasingly used as automatic judges for question answering (QA) and other reference-conditioned evaluation tasks, little is known about the…
Black-Box Hallucination Detection via Consistency Under the Uncertain Expression
Seongho Joo, Kyungmin Min, Jahyun Koo +1
Despite the great advancement of Language modeling in recent days, Large Language Models (LLMs) such as GPT3 are notorious for generating non-factual responses, so-called "hallucin…
Can You Trick the Grader? Adversarial Persuasion of LLM Judges
Yerin Hwang, Dongryeol Lee, Taegwan Kang +2
As large language models take on growing roles as automated evaluators in practical settings, a critical question arises: Can individuals persuade an LLM judge to assign unfairly h…
Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluation
Jiwon Moon, Yerin Hwang, Dongryeol Lee +3
With the growing use of large language models(LLMs) as evaluators, their application has expanded to code evaluation tasks, where they assess the correctness of generated code with…
Fooling the LVLM Judges: Visual Biases in LVLM-Based Evaluation
Yerin Hwang, Dongryeol Lee, Kyungmin Min +3
Recently, large vision-language models (LVLMs) have emerged as the preferred tools for judging text-image alignment, yet their robustness along the visual modality remains underexp…
Conditional [MASK] Discrete Diffusion Language Model
Hyukhun Koh, Minha Jhang, Dohyung Kim +2
Although auto-regressive models excel in natural language processing, they often struggle to generate diverse text and provide limited controllability. Non-auto-regressive methods…
Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluation
Dongryeol Lee, Yerin Hwang, Yongil Kim +2
In line with the principle of honesty, there has been a growing effort to train large language models (LLMs) to generate outputs containing epistemic markers. However, evaluation i…
Fine-grained Gender Control in Machine Translation with Large Language Models
Minwoo Lee, Hyukhun Koh, Minsung Kim +1
In machine translation, the problem of ambiguously gendered input has been pointed out, where the gender of an entity is not available in the source sentence. To address this ambig…
Return of EM: Entity-driven Answer Set Expansion for QA Evaluation
Dongryeol Lee, Minwoo Lee, Kyungmin Min +2
Recently, directly using large language models (LLMs) has been shown to be the most reliable method to evaluate QA models. However, it suffers from limited interpretability, high c…