1 citations · 1 across the 6 of their papers we have counts for
7 papers
RH-Detect: A Unified Benchmark for Reward Hacking Detection
Junwei Quan, Evgenii Opryshko, Rohan Subramani +1
Reward hacking, where a model exploits an evaluation signal without completing the intended task, threatens the reliability of deployed language model systems. Existing datasets us…
How does information access affect LLM monitors' ability to detect sabotage?
Rauno Arike, Raja Mehta Moreno, Rohan Subramani +2
Frontier language model agents can exhibit misaligned behaviors, including deception, exploiting reward hacks, and pursuing hidden objectives. To control potentially misaligned age…
Password-Activated Shutdown Protocols for Misaligned Frontier Agents
Kai Williams, Rohan Subramani, Francis Rhys Ward
Frontier AI developers may fail to align or control highly-capable AI agents. In many cases, it could be useful to have emergency shutdown mechanisms which effectively prevent misa…
Higher-Order Belief in Incomplete Information MAIDs
Jack Foxabbott, Rohan Subramani, Francis Rhys Ward
Multi-agent influence diagrams (MAIDs) are probabilistic graphical models which represent strategic interactions between agents. MAIDs are equivalent to extensive form games (EFGs)…
The Partially Observable Off-Switch Game
Andrew Garber, Rohan Subramani, Linus Luu +3
A wide variety of goals could cause an AI to disable its off switch because "you can't fetch the coffee if you're dead" (Russell 2019). Prior theoretical work on this shutdown prob…
Generalization Analogies: A Testbed for Generalizing AI Oversight to Hard-To-Measure Domains
Joshua Clymer, Garrett Baker, Rohan Subramani +1
As AI systems become more intelligent and their behavior becomes more challenging to assess, they may learn to game the flaws of human feedback instead of genuinely striving to fol…
On The Expressivity of Objective-Specification Formalisms in Reinforcement Learning
Rohan Subramani, Marcus Williams, Max Heitmann +3
Most algorithms in reinforcement learning (RL) require that the objective is formalised with a Markovian reward function. However, it is well-known that certain tasks cannot be exp…