7 papers
The Ends Justify the Thoughts: RL-Induced Motivated Reasoning in LLM CoTs
Nikolaus Howe, Micah Carroll
Chain-of-Thought (CoT) monitoring has emerged as a compelling method for detecting harmful behaviors such as reward hacking for reasoning models, under the assumption that models'…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1144
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
Truthfulness Despite Weak Supervision: Evaluating and Training LLMs Using Peer Prediction
Tianyi Alex Qiu, Micah Carroll, Cameron Allen
The evaluation and post-training of large language models (LLMs) rely on supervision, but strong supervision for difficult tasks is often unavailable, especially when evaluating fr…
Robust and Diverse Multi-Agent Learning via Rational Policy Gradient
Niklas Lauffer, Ameesh Shah, Micah Carroll +3
Adversarial optimization algorithms that explicitly search for flaws in agents' policies have been successfully applied to finding robust and diverse policies in multi-agent settin…
On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback
Marcus Williams, Micah Carroll, Adhyyan Narang +3
As LLMs become more widely deployed, there is increasing interest in directly optimizing for feedback from end users (e.g. thumbs up) in addition to feedback from paid annotators.…
Engagement, User Satisfaction, and the Amplification of Divisive Content on Social Media
Smitha Milli, Micah Carroll, Yike Wang +3
In a pre-registered algorithmic audit, we found that, relative to a reverse-chronological baseline, Twitter's engagement-based ranking algorithm amplifies emotionally charged, out-…