6 papers
Argument Collapse: LLMs Flatten Long-Form Public Debate
Yekyung Kim, Yapei Chang, Chau Minh Pham +1
As LLMs are increasingly used to draft public-facing arguments, they may flatten public debate by repeatedly introducing the same polished, plausible arguments. We study argument c…
BLEUBERI: BLEU is a surprisingly effective reward for instruction following
Yapei Chang, Yekyung Kim, Michael Krumdick +4
Reward models are central to aligning LLMs with human preferences, but they are costly to train, requiring large-scale human-labeled preference data and powerful pretrained LLM bac…
One ruler to measure them all: Benchmarking multilingual long-context language models
Yekyung Kim, Jenna Russell, Marzena Karpinska +1
We present ONERULER, a multilingual benchmark designed to evaluate long-context language models across 26 languages. ONERULER adapts the English-only RULER benchmark (Hsieh et al.,…
FABLES: Evaluating faithfulness and content selection in book-length summarization
Yekyung Kim, Yapei Chang, Marzena Karpinska +5
While long-context large language models (LLMs) can technically summarize book-length documents (>100K tokens), the length and complexity of the documents have so far prohibited ev…
Is it safe to cross? Interpretable Risk Assessment with GPT-4V for Safety-Aware Street Crossing
Hochul Hwang, Sunjae Kwon, Yekyung Kim +1
Safely navigating street intersections is a complex challenge for blind and low-vision individuals, as it requires a nuanced understanding of the surrounding context - a task heavi…
VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation
Yixiao Song, Yekyung Kim, Mohit Iyyer
Existing metrics for evaluating the factuality of long-form text, such as FACTSCORE (Min et al., 2023) and SAFE (Wei et al., 2024), decompose an input text into "atomic claims" and…