4 papers
The Promises and Limitations of Evaluation-Based AI Governance: Evidence from Expert Interviews
Lily Stelling, Jacob Charnock, Kevin L. Wei +1
We conducted 37 semi-structured expert interviews to examine how frontier AI evaluations reduce societal-level AI risks. Evaluations have become central to AI risk governance, desp…
Embedded Assessments for Frontier AI
Jacob Charnock, Sophie Williams, Zaheed Kara +5
Third-party evaluations for frontier AI have mostly tested models through external interfaces before deployment. But the risks from frontier AI models depend on how their developer…
What Should Frontier AI Developers Disclose About Internal Deployments?
Jacob Charnock, Raja Mehta Moreno, Justin Miller +1
Frontier AI developers are increasingly deploying highly capable models internally to automate AI R&D, but these deployments currently face limited external oversight. It is essent…
Expanding External Access To Frontier AI Models For Dangerous Capability Evaluations
Jacob Charnock, Alejandro Tlaie, Kyle O'Brien +2
Frontier AI companies increasingly rely on external evaluations to assess risks from dangerous capabilities before deployment. However, external evaluators often receive limited mo…