1 citations · 1 across the 3 of their papers we have counts for
4 papers
JudgeProfile: Understanding and Steering Subjectivity in LLM Judges
Qi Cao, Kangning Liu, Xuan Kan +10
LLM judges are inherently subjective, often favoring different responses in pairwise comparison when neither option is objectively wrong. To study this subjectivity, we introduce J…
Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge
Junjie Wu, Xuan Kan, Zihao He +3
Multimodal Large Language Models (MLLMs) have been widely adopted as MLLM-as-a-Judges due to their strong alignment with human judgment across various visual tasks. However, most e…
Bi-Level Prompt Optimization for Multimodal LLM-as-a-Judge
Bo Pan, Xuan Kan, Kaitai Zhang +6
Large language models (LLMs) have become widely adopted as automated judges for evaluating AI-generated content. Despite their success, aligning LLM-based evaluations with human ju…
Supporting Human Raters with the Detection of Harmful Content using Large Language Models
Kurt Thomas, Patrick Gage Kelley, David Tao +7
In this paper, we explore the feasibility of leveraging large language models (LLMs) to automate or otherwise assist human raters with identifying harmful content including hate sp…