Reasoning Under Pressure: How do Training Incentives Influence Chain-of-Thought Monitorability?

Matt MacDermott, Qiyao Wei, Rada Djoneva +1

AI systems that output their reasoning in natural language offer an opportunity for safety -- we can \emph{monitor} their chain of thought (CoT) for undesirable reasoning, such as…