3 citations · 8 across the 9 of their papers we have counts for
3 papers · 1 filter
PetriBench: Benchmarking LLM Reasoning over Dynamic State Spaces
Pyrros Koussios, Benjamin Jäger, John Hua Yao +3
Characterizing LLM reasoning remains an open challenge, as many existing benchmarks isolate specific reasoning skills, rely on external knowledge, or are costly to extend. We intro…
One Bias After Another: Mechanistic Reward Shaping and Persistent Biases in Language Reward Models
Daniel Fein, Max Lamparth, Violet Xiang +2
Reward Models (RMs) are crucial for online alignment of language models (LMs) with human preferences. However, RM-based preference-tuning is vulnerable to reward hacking, whereby L…
LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writing
Daniel Fein, Sebastian Russo, Violet Xiang +3
Evaluating creative writing generated by large language models (LLMs) remains challenging because open-ended narratives lack ground truths. Without performant automated evaluation…