495 citations · 958 across the 18 of their papers we have counts for
19 papers
Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
Tomek Korbak, Mikita Balesni, Elizabeth Barnes +38
AI systems that "think" in human language offer a unique opportunity for AI safety: we can monitor their chains of thought (CoT) for the intent to misbehave. Like all other known A…
Evaluating Frontier Models for Stealth and Situational Awareness
Mary Phuong, Roland S. Zimmermann, Ziyue Wang +6
Recent work has demonstrated the plausibility of frontier AI models scheming -- knowingly and covertly pursuing an objective misaligned with its developer's intentions. Such behavi…
A Framework for Evaluating Emerging Cyberattack Capabilities of AI
Mikel Rodriguez, Raluca Ada Popa, Four Flynn +3
As frontier AI models become more capable, evaluating their potential to enable cyberattacks is crucial for ensuring the safe development of Artificial General Intelligence (AGI).…
An Approach to Technical AGI Safety and Security
Rohin Shah, Alex Irpan, Alexander Matt Turner +27
Artificial General Intelligence (AGI) promises transformative benefits but also presents significant risks. We develop an approach to address the risk of harms consequential enough…
VISTA: A Visual and Textual Attention Dataset for Interpreting Multimodal Models
Harshit, Tolga Tasdizen
The recent developments in deep learning led to the integration of natural language processing (NLP) with computer vision, resulting in powerful integrated Vision and Language Mode…
The Ethics of Advanced AI Assistants
Iason Gabriel, Arianna Manzini, Geoff Keeling +54
This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural langu…
Holistic Safety and Responsibility Evaluations of Advanced AI Models
Laura Weidinger, Joslyn Barnhart, Jenny Brennan +16
Safety and responsibility evaluations of advanced AI models are a critical but developing field of research and practice. In the development of Google DeepMind's advanced AI models…
Evaluating Frontier Models for Dangerous Capabilities
Mary Phuong, Matthew Aitchison, Elliot Catt +24
To understand the risks posed by a new AI system, we must understand what it can and cannot do. Building on prior work, we introduce a programme of new "dangerous capability" evalu…
Levels of AGI for Operationalizing Progress on the Path to AGI
Meredith Ringel Morris, Jascha Sohl-Dickstein, Noah Fiedel +5
We propose a framework for classifying the capabilities and behavior of Artificial General Intelligence (AGI) models and their precursors. This framework introduces levels of AGI p…
International Institutions for Advanced AI
Lewis Ho, Joslyn Barnhart, Robert Trager +8
International institutions may have an important role to play in ensuring advanced AI systems benefit humanity. International collaborations can unlock AI's ability to further sust…
Model evaluation for extreme risks
Toby Shevlane, Sebastian Farquhar, Ben Garfinkel +18
Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities. Further progress in AI development could lead to ca…
Democratising AI: Multiple Meanings, Goals, and Methods
Elizabeth Seger, Aviv Ovadya, Ben Garfinkel +2
Numerous parties are calling for the democratisation of AI, but the phrase is used to refer to a variety of goals, the pursuit of which sometimes conflict. This paper identifies fo…
Normative Disagreement as a Challenge for Cooperative AI
Julian Stastny, Maxime Riché, Alexander Lyzhov +3
Cooperation in settings where agents have both common and conflicting interests (mixed-motive environments) has recently received considerable attention in multi-agent learning. Ho…
Beyond Privacy Trade-offs with Structured Transparency
Andrew Trask, Emma Bluemke, Teddy Collins +5
Successful collaboration involves sharing information. However, parties may disagree on how the information they need to share should be used. We argue that many of these concerns…
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
Miles Brundage, Shahar Avin, Jasmine Wang +56
With the recent wave of progress in artificial intelligence (AI) has come a growing awareness of the large-scale impacts of AI systems, and recognition that existing regulations an…
The Windfall Clause: Distributing the Benefits of AI for the Common Good
Cullen O'Keefe, Peter Cihon, Ben Garfinkel +3
As the transformative potential of AI has become increasingly salient as a matter of public and political interest, there has been growing discussion about the need to ensure that…
The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse?
Toby Shevlane, Allan Dafoe
There is growing concern over the potential misuse of artificial intelligence (AI) research. Publishing scientific research can facilitate misuse of the technology, but the researc…
Between Progress and Potential Impact of AI: the Neglected Dimensions
Fernando Martínez-Plumed, Shahar Avin, Miles Brundage +3
We reframe the analysis of progress in AI by incorporating into an overall framework both the task performance of a system, and the time and resource costs incurred in the developm…
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Miles Brundage, Shahar Avin, Jack Clark +23
This report surveys the landscape of potential security threats from malicious uses of AI, and proposes ways to better forecast, prevent, and mitigate these threats. After analyzin…