24 citations · 37 across the 15 of their papers we have counts for
4 papers · 1 filter
AI Control: Improving Safety Despite Intentional Subversion
Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan +1
As large language models (LLMs) become more powerful and are deployed more autonomously, it will be increasingly important to prevent them from causing harmful outcomes. Researcher…
Preventing Language Models From Hiding Their Reasoning
Fabien Roger, Ryan Greenblatt
Large language models (LLMs) often benefit from intermediate steps of reasoning to generate answers to complex problems. When these intermediate steps of reasoning are used to moni…
Benchmarks for Detecting Measurement Tampering
Fabien Roger, Ryan Greenblatt, Max Nadeau +2
When training powerful AI systems to perform complex tasks, it may be challenging to provide training signals which are robust to optimization. One concern is \textit{measurement t…
Large Language Models Sometimes Generate Purely Negatively-Reinforced Text
Fabien Roger
When using adversarial training, it is common practice to train against the most egregious failures. However, this might imply using examples with sensitive information (such as le…