45 citations · 53 across the 5 of their papers we have counts for
5 papers
Modeling Transformative AI Risks (MTAIR) Project -- Summary Report
Sam Clarke, Ben Cottier, Aryeh Englander +4
This report outlines work by the Modeling Transformative AI Risk (MTAIR) project, an attempt to map out the key hypotheses, uncertainties, and disagreements in debates about catast…
Arguments about Highly Reliable Agent Designs as a Useful Path to Artificial Intelligence Safety
Issa Rice, David Manheim
Several different approaches exist for ensuring the safety of future Transformative Artificial Intelligence (TAI) or Artificial Superintelligence (ASI) systems, and proponents of d…
Oversight of Unsafe Systems via Dynamic Safety Envelopes
David Manheim
This paper reviews the reasons that Human-in-the-Loop is both critical for preventing widely-understood failure modes for machine learning, and not a practical solution. Following…
Multiparty Dynamics and Failure Modes for Machine Learning and Artificial Intelligence
David Manheim
An important challenge for safety in machine learning and artificial intelligence systems is a~set of related failures involving specification gaming, reward hacking, fragility to…
Categorizing Variants of Goodhart's Law
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics. This occurs when a metric which can be used to improve a system is used to an exte…