124 citations · 437 across the 18 of their papers we have counts for
11 papers · 1 filter
Truthfulness Despite Weak Supervision: Evaluating and Training LLMs Using Peer Prediction
Tianyi Alex Qiu, Micah Carroll, Cameron Allen
The evaluation and post-training of large language models (LLMs) rely on supervision, but strong supervision for difficult tasks is often unavailable, especially when evaluating fr…
The Ends Justify the Thoughts: RL-Induced Motivated Reasoning Fools CoT Monitors
Nikolaus Howe, Micah Carroll
Chain-of-thought (CoT) monitoring has emerged as a compelling method for detecting harmful behaviors such as reward hacking in reasoning models, under the assumption that models' r…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1168
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback
Marcus Williams, Micah Carroll, Adhyyan Narang +3
As LLMs become more widely deployed, there is increasing interest in directly optimizing for feedback from end users (e.g. thumbs up) in addition to feedback from paid annotators.…
Time-Efficient Reward Learning via Visually Assisted Cluster Ranking
David Zhang, Micah Carroll, Andreea Bobu +1
One of the most successful paradigms for reward learning uses human feedback in the form of comparisons. Although these methods hold promise, human comparison labeling is expensive…
UniMASK: Unified Inference in Sequential Decision Problems
Micah Carroll, Orr Paradise, Jessy Lin +8
Randomly masking and predicting word tokens has been a successful approach in pre-training language models for a variety of downstream tasks. In this work, we observe that the same…
Optimal Behavior Prior: Data-Efficient Human Models for Improved Human-AI Collaboration
Mesut Yang, Micah Carroll, Anca Dragan
AI agents designed to collaborate with people benefit from models that enable them to anticipate human behavior. However, realistic models tend to require vast amounts of human dat…
Towards Flexible Inference in Sequential Decision Problems via Bidirectional Transformers
Micah Carroll, Jessy Lin, Orr Paradise +8
Randomly masking and predicting word tokens has been a successful approach in pre-training language models for a variety of downstream tasks. In this work, we observe that the same…
Estimating and Penalizing Induced Preference Shifts in Recommender Systems
Micah Carroll, Anca Dragan, Stuart Russell +1
The content that a recommender system (RS) shows to users influences them. Therefore, when choosing a recommender to deploy, one is implicitly also choosing to induce specific inte…
Evaluating the Robustness of Collaborative Agents
Paul Knott, Micah Carroll, Sam Devlin +4
In order for agents trained by deep reinforcement learning to work alongside humans in realistic settings, we will need to ensure that the agents are \emph{robust}. Since the real…
On the Utility of Learning about Humans for Human-AI Coordination
Micah Carroll, Rohin Shah, Mark K. Ho +4
While we would like agents that can coordinate with humans, current algorithms such as self-play and population-based training create agents that can coordinate with themselves. Ag…