281 citations · 525 across the 15 of their papers we have counts for
15 papers
Modular Pretraining Enables Access Control
Ethan Roland, Murat Cubuktepe, Erick Martinez +8
AI developers face a dual-use dilemma. An AI capability that helps one user cure a disease can help another synthesize one. This dilemma could be resolved with access control, limi…
Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs
Igor Shilov, Alex Cloud, Aryo Pradipta Gema +5
Large Language Models increasingly possess capabilities that carry dual-use risks. While data filtering has emerged as a pretraining-time mitigation, it faces significant challenge…
Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
Mrinank Sharma, Meg Tong, Jesse Mu +40
Large language models (LLMs) are vulnerable to universal jailbreaks-prompting strategies that systematically bypass model safeguards and enable users to carry out harmful processes…
Sabotage Evaluations for Frontier Models
Joe Benton, Misha Wagner, Eric Christiansen +13
Sufficiently capable models could subvert human oversight and decision-making in important contexts. For example, in the context of AI development, models could covertly sabotage e…
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
Evan Hubinger, Carson Denison, Jesse Mu +36
Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives when giv…
Studying Large Language Model Generalization with Influence Functions
Roger Grosse, Juhan Bae, Cem Anil +14
When trying to gain better visibility into a machine learning model in order to understand and mitigate the associated risks, a potentially valuable source of evidence is: which tr…
Path Independent Equilibrium Models Can Better Exploit Test-Time Computation
Cem Anil, Ashwini Pokle, Kaiqu Liang +5
Designing networks capable of attaining better performance with an increased inference budget is important to facilitate generalization to harder problem instances. Recent efforts…
Exploring Length Generalization in Large Language Models
Cem Anil, Yuhuai Wu, Anders Andreassen +7
The ability to extrapolate from short problem instances to longer ones is an important form of out-of-distribution generalization in reasoning tasks, and is crucial when learning f…
Solving Quantitative Reasoning Problems with Language Models
Aitor Lewkowycz, Anders Andreassen, David Dohan +11
Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. Nevertheless, state-of-the-art models have generally stru…
Learning to Elect
Cem Anil, Xuchan Bao
Voting systems have a wide range of applications including recommender systems, web search, product design and elections. Limited by the lack of general-purpose analytical tools, i…
Learning to Give Checkable Answers with Prover-Verifier Games
Cem Anil, Guodong Zhang, Yuhuai Wu +1
Our ability to know when to trust the decisions made by machine learning systems has not kept up with the staggering improvements in their performance, limiting their applicability…
Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networks
Qiyang Li, Saminul Haque, Cem Anil +3
Lipschitz constraints under L2 norm on deep neural networks are useful for provable adversarial robustness bounds, stable training, and Wasserstein distance estimation. While heuri…
TimbreTron: A WaveNet(CycleGAN(CQT(Audio))) Pipeline for Musical Timbre Transfer
Sicong Huang, Qiyang Li, Cem Anil +3
In this work, we address the problem of musical timbre transfer, where the goal is to manipulate the timbre of a sound sample from one instrument to match another instrument while…
Sorting out Lipschitz function approximation
Cem Anil, James Lucas, Roger Grosse
Training neural networks under a strict Lipschitz constraint is useful for provable adversarial robustness, generalization bounds, interpretable gradients, and Wasserstein distance…
Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization
Jonathan Tremblay, Aayush Prakash, David Acuna +7
We present a system for training deep neural networks for object detection using synthetic images. To handle the variability in real-world data, the system relies upon the techniqu…