215 citations · 868 across the 14 of their papers we have counts for
3 papers · 1 filter
Monitoring Monitorability
Melody Y. Guan, Miles Wang, Micah Carroll +9
Observability into the decision making of modern AI systems may be required to safely deploy increasingly capable agents. Monitoring the chain-of-thought (CoT) of today's reasoning…
Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
Bowen Baker, Joost Huizinga, Leo Gao +6
Mitigating reward hacking--where AI systems misbehave due to flaws or misspecifications in their learning objectives--remains a key challenge in constructing capable and aligned mo…
First return, then explore
Adrien Ecoffet, Joost Huizinga, Joel Lehman +2
The promise of reinforcement learning is to solve complex sequential decision problems autonomously by specifying a high-level reward function only. However, reinforcement learning…