3.2k citations · 9.8k across the 51 of their papers we have counts for
23 papers · 1 filter
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
Nathaniel Li, Alexander Pan, Anjali Gopal +54
The White House Executive Order on Artificial Intelligence highlights the risks of large language models (LLMs) empowering malicious actors in developing biological, cyber, and che…
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Mantas Mazeika, Long Phan, Xuwang Yin +9
Automated red teaming holds substantial promise for uncovering and mitigating the risks associated with the malicious use of large language models (LLMs), yet the field lacks a sta…
Representation Engineering: A Top-Down Approach to AI Transparency
Andy Zou, Long Phan, Sarah Chen +18
In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights f…
Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark
Alexander Pan, Jun Shern Chan, Andy Zou +7
Artificial agents have traditionally been trained to maximize reward, which may incentivize power-seeking and deception, analogous to how next-token prediction in language models (…
Forecasting Future World Events with Neural Networks
Andy Zou, Tristan Xiao, Ryan Jia +7
Forecasting future world events is a challenging but valuable task. Forecasts of climate, geopolitical conflict, pandemics and economic indicators help shape policy and decision ma…
PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
Dan Hendrycks, Andy Zou, Mantas Mazeika +4
In real-world applications of machine learning, reliable and safe systems must consider measures of performance beyond standard test set accuracy. These other goals include out-of-…
What Would Jiminy Cricket Do? Towards Agents That Behave Morally
Dan Hendrycks, Mantas Mazeika, Andy Zou +6
When making everyday decisions, people are guided by their conscience, an internal sense of right and wrong. By contrast, artificial agents are currently not endowed with a moral s…
Unsolved Problems in ML Safety
Dan Hendrycks, Nicholas Carlini, John Schulman +1
Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings. As with other powerful technolo…
VisDA-2021 Competition Universal Domain Adaptation to Improve Performance on Out-of-Distribution Data
Dina Bashkirova, Dan Hendrycks, Donghyun Kim +5
Progress in machine learning is typically measured by training and testing a model on the same distribution of data, i.e., the same domain. This over-estimates future accuracy on o…
Measuring Mathematical Problem Solving With the MATH Dataset
Dan Hendrycks, Collin Burns, Saurav Kadavath +5
Many intellectual endeavors require mathematical problem solving, but this skill remains beyond the capabilities of computers. To measure this ability in machine learning models, w…
Testing Robustness Against Unforeseen Adversaries
Max Kaufmann, Daniel Kang, Yi Sun +9
Adversarial robustness research primarily focuses on L_p perturbations, and most defenses are developed with identical training-time and test-time adversaries. However, in real-wor…
Natural Adversarial Examples
Dan Hendrycks, Kevin Zhao, Steven Basart +2
We introduce two challenging datasets that reliably cause machine learning model performance to substantially degrade. The datasets are collected with a simple adversarial filtrati…
Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath +1
Self-supervision provides effective representations for downstream tasks without requiring labels. However, existing approaches lag behind fully supervised training and are often n…
Transfer of Adversarial Robustness Between Perturbation Types
Daniel Kang, Yi Sun, Tom Brown +2
We study the transfer of adversarial robustness of deep neural networks between different perturbation types. While most work on adversarial examples has focused on and…
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
Dan Hendrycks, Thomas Dietterich
In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while sho…
Deep Anomaly Detection with Outlier Exposure
Dan Hendrycks, Mantas Mazeika, Thomas Dietterich
It is important to detect anomalous inputs when deploying machine learning systems. The use of larger and more complex inputs in deep learning magnifies the difficulty of distingui…
Using Pre-Training Can Improve Model Robustness and Uncertainty
Dan Hendrycks, Kimin Lee, Mantas Mazeika
He et al. (2018) have called into question the utility of pre-training by showing that training from scratch can often yield similar performance to pre-training. We show that altho…
Open Category Detection with PAC Guarantees
Si Liu, Risheek Garrepalli, Thomas G. Dietterich +2
Open category detection is the problem of detecting "alien" test instances that belong to categories or classes that were not present in the training data. In many applications, re…
Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
Dan Hendrycks, Thomas G. Dietterich
In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while sho…
Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
Dan Hendrycks, Mantas Mazeika, Duncan Wilson +1
The growing importance of massive datasets used for deep learning makes robustness to label noise a critical property for classifiers to have. Sources of label noise include automa…
Early Methods for Detecting Adversarial Images
Dan Hendrycks, Kevin Gimpel
Many machine learning classifiers are vulnerable to adversarial perturbations. An adversarial perturbation modifies an input to change a classifier's prediction without causing the…
Adjusting for Dropout Variance in Batch Normalization and Weight Initialization
Dan Hendrycks, Kevin Gimpel
We show how to adjust for the variance introduced by dropout with corrections to weight initialization and Batch Normalization, yielding higher accuracy. Though dropout can preserv…
Gaussian Error Linear Units (GELUs)
Dan Hendrycks, Kevin Gimpel
We propose the Gaussian Error Linear Unit (GELU), a high-performing neural network activation function. The GELU activation function is , where the standard Gaussian…