307 citations · 329 across the 6 of their papers we have counts for
7 papers
Comprehension Audits to Mitigate Risks from Automated AI Research
Ronald J. Bodkin, Bahrad A. Sokhansanj, Gillian K. Hadfield
AI is already writing a majority of code for frontier AI labs. This creates a safety risk if there is insufficient human oversight. Existing work proposes minimum comprehension thr…
Can AI agents conduct open-ended AI research? Early evidence from two case studies
Peter Kirgis, Sayash Kapoor, Andrew Schwartz +21
Forecasts of explosive AI progress hinge on AI agents automating AI research. But evidence on whether agents can carry out open-ended AI research is thin. Current evaluations eithe…
Open-World Evaluations for Measuring Frontier AI Capabilities
Sayash Kapoor, Peter Kirgis, Andrew Schwartz +15
Benchmark-based evaluation remains important for tracking frontier AI progress. But it can both overstate and understate deployed capability because it privileges tasks that can be…
I-CALM: Incentivizing Confidence-Aware Abstention for LLM Selective Answering
Haotian Zong, Binze Li, Yufei Long +3
Large language models (LLMs) often produce confident but incorrect answers, in part because standard evaluation incentives reward guessing over expressing uncertainty. We study epi…
Responsible Reporting for Frontier AI Development
Noam Kolt, Markus Anderljung, Joslyn Barnhart +7
Mitigating the risks from frontier AI systems requires up-to-date and reliable information about those systems. Organizations that develop and deploy frontier systems have signific…
Computing Power and the Governance of Artificial Intelligence
Girish Sastry, Lennart Heim, Haydn Belfield +16
Computing power, or "compute," is crucial for the development and deployment of artificial intelligence (AI) capabilities. As a result, governments and companies have started to le…
Managing extreme AI risks amid rapid progress
Yoshua Bengio, Geoffrey Hinton, Andrew Yao +22
Artificial Intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act and pursue goals. Increase…