most citedBlack-Box Access is Insufficient for Rigorous AI Audits

72 citations · 79 across the 5 of their papers we have counts for

collaborators

5 papers

cs.LG2025

Focused Skill Discovery: Learning to Control Specific State Variables while Minimizing Side Effects

Jonathan Colaço Carr, Qinyi Sun, Cameron Allen

Skills are essential for unlocking higher levels of problem solving. A common approach to discovering these skills is to learn ones that reliably reach different states, thus empow…

cs.LG2025

Fractal Generative Models

Tianhong Li, Qinyi Sun, Lijie Fan +1

Modularization is a cornerstone of computer science, abstracting complex functions into atomic building blocks. In this paper, we introduce a new level of modularization by abstrac…

cs.SE2024★ 5 cited

DafnyBench: A Benchmark for Formal Software Verification

Chloe Loughridge, Qinyi Sun, Seth Ahrenbach +7

We introduce DafnyBench, the largest benchmark of its kind for training and evaluating machine learning systems for formal software verification. We test the ability of LLMs such a…

cs.LG2024★ 2 cited

Universal Neurons in GPT2 Language Models

Wes Gurnee, Theo Horsley, Zifan Carl Guo +5

A basic question within the emerging field of mechanistic interpretability is the degree to which neural networks learn the same underlying mechanisms. In other words, are neural m…

cs.CY2024★ 72 cited

Black-Box Access is Insufficient for Rigorous AI Audits

Stephen Casper, Carson Ezell, Charlotte Siegmann +18

External audits of AI systems are increasingly recognized as a key mechanism for AI governance. The effectiveness of an audit, however, depends on the degree of access granted to a…