1 citations · 2 across the 12 of their papers we have counts for
12 papers
Language Shapes Instruction Hierarchy Compliance in Multilingual LLMs
Jiwon Moon, Yerin Hwang, Kyomin Jung
Instruction hierarchy (IH) requires models to prioritize instructions by source, ensuring that higher-priority instructions override lower-priority ones. Despite its importance for…
When Wording Steers the Evaluation: Framing Bias in LLM judges
Yerin Hwang, Dongryeol Lee, Taegwan Kang +2
Large language models (LLMs) are known to produce varying responses depending on prompt phrasing, indicating that subtle guidance in phrasing can steer their answers. However, the…
Judging Against the Reference: Uncovering Knowledge-Driven Failures in LLM-Judges on QA Evaluation
Dongryeol Lee, Yerin Hwang, Taegwan Kang +3
While large language models (LLMs) are increasingly used as automatic judges for question answering (QA) and other reference-conditioned evaluation tasks, little is known about the…
Can You Trick the Grader? Adversarial Persuasion of LLM Judges
Yerin Hwang, Dongryeol Lee, Taegwan Kang +2
As large language models take on growing roles as automated evaluators in practical settings, a critical question arises: Can individuals persuade an LLM judge to assign unfairly h…
Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluation
Jiwon Moon, Yerin Hwang, Dongryeol Lee +3
With the growing use of large language models(LLMs) as evaluators, their application has expanded to code evaluation tasks, where they assess the correctness of generated code with…
Fooling the LVLM Judges: Visual Biases in LVLM-Based Evaluation
Yerin Hwang, Dongryeol Lee, Kyungmin Min +3
Recently, large vision-language models (LVLMs) have emerged as the preferred tools for judging text-image alignment, yet their robustness along the visual modality remains underexp…
LLMs can be easily Confused by Instructional Distractions
Yerin Hwang, Yongil Kim, Jahyun Koo +3
Despite the fact that large language models (LLMs) show exceptional skill in instruction following tasks, this strength can turn into a vulnerability when the models are required t…
Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluation
Dongryeol Lee, Yerin Hwang, Yongil Kim +2
In line with the principle of honesty, there has been a growing effort to train large language models (LLMs) to generate outputs containing epistemic markers. However, evaluation i…
SWITCH: Studying with Teacher for Knowledge Distillation of Large Language Models
Jahyun Koo, Yerin Hwang, Yongil Kim +3
Despite the success of Large Language Models (LLMs), they still face challenges related to high inference costs and memory requirements. To address these issues, Knowledge Distilla…
MP2D: An Automated Topic Shift Dialogue Generation Framework Leveraging Knowledge Graphs
Yerin Hwang, Yongil Kim, Yunah Jang +3
Despite advancements in on-topic dialogue systems, effectively managing topic shifts within dialogues remains a persistent challenge, largely attributed to the limited availability…
Dialogizer: Context-aware Conversational-QA Dataset Generation from Textual Sources
Yerin Hwang, Yongil Kim, Hyunkyung Bae +3
To address the data scarcity issue in Conversational question answering (ConvQA), a dialog inpainting method, which utilizes documents to generate ConvQA datasets, has been propose…
PR-MCS: Perturbation Robust Metric for MultiLingual Image Captioning
Yongil Kim, Yerin Hwang, Hyeongu Yun +3
Vulnerability to lexical perturbation is a critical weakness of automatic evaluation metrics for image captioning. This paper proposes Perturbation Robust Multi-Lingual CLIPScore(P…