Distillation for Incrimination and Distillation for Capabilities

Sebastian Prasanna, Jacqueline Tay, Alek Westover

Powerful misaligned AI models might recognize alignment evaluations and strategically behave well on them, rendering direct audits uninformative. However, distilling such a model i…