219 citations · 658 across the 34 of their papers we have counts for
28 papers · 1 filter
Embedded Assessments for Frontier AI
Jacob Charnock, Sophie Williams, Zaheed Kara +5
Third-party evaluations for frontier AI have mostly tested models through external interfaces before deployment. But the risks from frontier AI models depend on how their developer…
Underwriting the Agent Economy: The Blueprint for an AI Insurance Stack
Cristian Trout, Sanmi Koyejo, Sasha Romanosky +34
From maritime trade to commercial nuclear power, insurance has been the enabler of major economic and technological developments by pricing risk, limiting downside, and spreading b…
Measuring AI R&D Automation
Alan Chan, Ranay Padarath, Joe Kwon +2
The automation of AI R&D (AIRDA) could have significant implications, but its extent and ultimate effects remain uncertain. We need empirical data to resolve these uncertainties, b…
Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies
Miles Brundage, Noemi Dreksler, Aidan Homewood +45
We outline a vision for frontier AI auditing, which we define as rigorous third-party verification of frontier AI developers' safety and security claims, and evaluation of their sy…
The Future of the AI Summit Series
Lucia Velasco, Charles Martinet, Henry de Zoete +27
This policy memo examines the evolution of the international AI Summit series, initiated at Bletchley Park in 2023 and continued through Seoul in 2024 and Paris in 2025, as a forum…
From Turing to Tomorrow: The UK's Approach to AI Regulation
Oliver Ritchie, Markus Anderljung, Tom Rachman
The UK has pursued a distinctive path in AI regulation: less cautious than the EU but more willing to address risks than the US, and has emerged as a global leader in coordinating…
On Regulating Downstream AI Developers
Sophie Williams, Jonas Schuett, Markus Anderljung
Foundation models - models trained on broad data that can be adapted to a wide range of downstream tasks - can pose significant risks, ranging from intimate image abuse, cyberattac…
Safety cases for frontier AI
Marie Davidsen Buhl, Gaurav Sett, Leonie Koessler +2
As frontier artificial intelligence (AI) systems become more capable, it becomes more important that developers can explain why their systems are sufficiently safe. One way to do s…
A Grading Rubric for AI Safety Frameworks
Jide Alaga, Jonas Schuett, Markus Anderljung
Over the past year, artificial intelligence (AI) companies have been increasingly adopting AI safety frameworks. These frameworks outline how companies intend to keep the potential…
Open Problems in Technical AI Governance
Anka Reuel, Ben Bucknall, Stephen Casper +30
AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at…
From Principles to Rules: A Regulatory Approach for Frontier AI
Jonas Schuett, Markus Anderljung, Alexis Carlier +2
Several jurisdictions are starting to regulate frontier artificial intelligence (AI) systems, i.e. general-purpose AI systems that match or exceed the capabilities present in the m…
Risk thresholds for frontier AI
Leonie Koessler, Jonas Schuett, Markus Anderljung
Frontier artificial intelligence (AI) systems could pose increasing risks to public safety and security. But what level of risk is acceptable? One increasingly popular approach is…
Towards interactive evaluations for interaction harms in human-AI systems
Lujain Ibrahim, Saffron Huang, Umang Bhatt +2
Current AI evaluation methods, which rely on static, model-only tests, fail to account for harms that emerge through sustained human-AI interaction. As AI systems proliferate and a…
Societal Adaptation to Advanced AI
Jamie Bernardi, Gabriel Mukobi, Hilary Greaves +2
Existing strategies for managing risks from advanced AI systems often focus on affecting what AI systems are developed and how they diffuse. However, this approach becomes less fea…
Responsible Reporting for Frontier AI Development
Noam Kolt, Markus Anderljung, Joslyn Barnhart +7
Mitigating the risks from frontier AI systems requires up-to-date and reliable information about those systems. Organizations that develop and deploy frontier systems have signific…
Computing Power and the Governance of Artificial Intelligence
Girish Sastry, Lennart Heim, Haydn Belfield +16
Computing power, or "compute," is crucial for the development and deployment of artificial intelligence (AI) capabilities. As a result, governments and companies have started to le…
Visibility into AI Agents
Alan Chan, Carson Ezell, Max Kaufmann +9
Increased delegation of commercial, scientific, governmental, and personal activities to AI agents -- systems capable of pursuing complex goals with limited supervision -- may exac…
Towards Publicly Accountable Frontier LLMs: Building an External Scrutiny Ecosystem under the ASPIRE Framework
Markus Anderljung, Everett Thornton Smith, Joe O'Brien +7
With the increasing integration of frontier large language models (LLMs) into society and the economy, decisions related to their training, deployment, and use have far-reaching im…
Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives
Elizabeth Seger, Noemi Dreksler, Richard Moulange +19
Recent decisions by leading AI labs to either open-source their models or to restrict access to their models has sparked debate about whether, and how, increasingly capable AI mode…
Frontier AI Regulation: Managing Emerging Risks to Public Safety
Markus Anderljung, Joslyn Barnhart, Anton Korinek +21
Advanced AI models hold the promise of tremendous benefits for humanity, but society needs to proactively manage the accompanying risks. In this paper, we focus on what we term "fr…
Towards best practices in AGI safety and governance: A survey of expert opinion
Jonas Schuett, Noemi Dreksler, Markus Anderljung +4
A number of leading AI companies, including OpenAI, Google DeepMind, and Anthropic, have the stated goal of building artificial general intelligence (AGI) - AI systems that achieve…
The Brussels Effect and Artificial Intelligence: How EU regulation will impact the global AI market
Charlotte Siegmann, Markus Anderljung
The European Union is likely to introduce among the first, most stringent, and most comprehensive AI regulatory regimes of the world's major jurisdictions. In this report, we ask w…
Forecasting AI Progress: Evidence from a Survey of Machine Learning Researchers
Baobao Zhang, Noemi Dreksler, Markus Anderljung +4
Advances in artificial intelligence (AI) are shaping modern life, from transportation, health care, science, finance, to national defense. Forecasts of AI development could help im…
Institutionalising Ethics in AI through Broader Impact Requirements
Carina Prunkl, Carolyn Ashurst, Markus Anderljung +3
Turning principles into practice is one of the most pressing challenges of artificial intelligence (AI) governance. In this article, we reflect on a novel governance initiative by…