219 citations · 658 across the 34 of their papers we have counts for
35 papers
Embedded Assessments for Frontier AI
Jacob Charnock, Sophie Williams, Zaheed Kara +5
Third-party evaluations for frontier AI have mostly tested models through external interfaces before deployment. But the risks from frontier AI models depend on how their developer…
Underwriting the Agent Economy: The Blueprint for an AI Insurance Stack
Cristian Trout, Sanmi Koyejo, Sasha Romanosky +34
From maritime trade to commercial nuclear power, insurance has been the enabler of major economic and technological developments by pricing risk, limiting downside, and spreading b…
Measuring AI R&D Automation
Alan Chan, Ranay Padarath, Joe Kwon +2
The automation of AI R&D (AIRDA) could have significant implications, but its extent and ultimate effects remain uncertain. We need empirical data to resolve these uncertainties, b…
Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies
Miles Brundage, Noemi Dreksler, Aidan Homewood +45
We outline a vision for frontier AI auditing, which we define as rigorous third-party verification of frontier AI developers' safety and security claims, and evaluation of their sy…
The Future of the AI Summit Series
Lucia Velasco, Charles Martinet, Henry de Zoete +27
This policy memo examines the evolution of the international AI Summit series, initiated at Bletchley Park in 2023 and continued through Seoul in 2024 and Paris in 2025, as a forum…
From Turing to Tomorrow: The UK's Approach to AI Regulation
Oliver Ritchie, Markus Anderljung, Tom Rachman
The UK has pursued a distinctive path in AI regulation: less cautious than the EU but more willing to address risks than the US, and has emerged as a global leader in coordinating…
In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI
Shayne Longpre, Kevin Klyman, Ruth E. Appel +31
The widespread deployment of general-purpose AI (GPAI) systems introduces significant new risks. Yet the infrastructure, practices, and norms for reporting flaws in GPAI systems re…
On Regulating Downstream AI Developers
Sophie Williams, Jonas Schuett, Markus Anderljung
Foundation models - models trained on broad data that can be adapted to a wide range of downstream tasks - can pose significant risks, ranging from intimate image abuse, cyberattac…
Infrastructure for AI Agents
Alan Chan, Kevin Wei, Sihao Huang +5
AI agents plan and execute interactions in open-ended environments. For example, OpenAI's Operator can use a web browser to do product comparisons and buy online goods. Much resear…
Safety cases for frontier AI
Marie Davidsen Buhl, Gaurav Sett, Leonie Koessler +2
As frontier artificial intelligence (AI) systems become more capable, it becomes more important that developers can explain why their systems are sufficiently safe. One way to do s…
A Grading Rubric for AI Safety Frameworks
Jide Alaga, Jonas Schuett, Markus Anderljung
Over the past year, artificial intelligence (AI) companies have been increasingly adopting AI safety frameworks. These frameworks outline how companies intend to keep the potential…
Open Problems in Technical AI Governance
Anka Reuel, Ben Bucknall, Stephen Casper +30
AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at…
From Principles to Rules: A Regulatory Approach for Frontier AI
Jonas Schuett, Markus Anderljung, Alexis Carlier +2
Several jurisdictions are starting to regulate frontier artificial intelligence (AI) systems, i.e. general-purpose AI systems that match or exceed the capabilities present in the m…
Risk thresholds for frontier AI
Leonie Koessler, Jonas Schuett, Markus Anderljung
Frontier artificial intelligence (AI) systems could pose increasing risks to public safety and security. But what level of risk is acceptable? One increasingly popular approach is…
IDs for AI Systems
Alan Chan, Noam Kolt, Peter Wills +7
AI systems are increasingly pervasive, yet information needed to decide whether and how to engage with them may not exist or be accessible. A user may not be able to verify whether…
Towards interactive evaluations for interaction harms in human-AI systems
Lujain Ibrahim, Saffron Huang, Umang Bhatt +2
Current AI evaluation methods, which rely on static, model-only tests, fail to account for harms that emerge through sustained human-AI interaction. As AI systems proliferate and a…
Societal Adaptation to Advanced AI
Jamie Bernardi, Gabriel Mukobi, Hilary Greaves +2
Existing strategies for managing risks from advanced AI systems often focus on affecting what AI systems are developed and how they diffuse. However, this approach becomes less fea…
Responsible Reporting for Frontier AI Development
Noam Kolt, Markus Anderljung, Joslyn Barnhart +7
Mitigating the risks from frontier AI systems requires up-to-date and reliable information about those systems. Organizations that develop and deploy frontier systems have signific…
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Usman Anwar, Abulhair Saparov, Javier Rando +39
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs). These challenges are organized into three different categories…
Computing Power and the Governance of Artificial Intelligence
Girish Sastry, Lennart Heim, Haydn Belfield +16
Computing power, or "compute," is crucial for the development and deployment of artificial intelligence (AI) capabilities. As a result, governments and companies have started to le…
Visibility into AI Agents
Alan Chan, Carson Ezell, Max Kaufmann +9
Increased delegation of commercial, scientific, governmental, and personal activities to AI agents -- systems capable of pursuing complex goals with limited supervision -- may exac…
Towards Publicly Accountable Frontier LLMs: Building an External Scrutiny Ecosystem under the ASPIRE Framework
Markus Anderljung, Everett Thornton Smith, Joe O'Brien +7
With the increasing integration of frontier large language models (LLMs) into society and the economy, decisions related to their training, deployment, and use have far-reaching im…
Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives
Elizabeth Seger, Noemi Dreksler, Richard Moulange +19
Recent decisions by leading AI labs to either open-source their models or to restrict access to their models has sparked debate about whether, and how, increasingly capable AI mode…
Frontier AI Regulation: Managing Emerging Risks to Public Safety
Markus Anderljung, Joslyn Barnhart, Anton Korinek +21
Advanced AI models hold the promise of tremendous benefits for humanity, but society needs to proactively manage the accompanying risks. In this paper, we focus on what we term "fr…