Certifying Confidence via Randomized Smoothing

Abstract

Randomized smoothing has been shown to provide good certified-robustness guarantees for high-dimensional classification problems. It uses the probabilities of predicting the top two most-likely classes around an input point under a smoothing distribution to generate a certified radius for a classifier's prediction. However, most smoothing methods do not give us any information about the confidence with which the underlying classifier (e.g., deep neural network) makes a prediction. In this work, we propose a method to generate certified radii for the prediction confidence of the smoothed classifier. We consider two notions for quantifying confidence: average prediction score of a class and the margin by which the average prediction score of one class exceeds that of another. We modify the Neyman-Pearson lemma (a key theorem in randomized smoothing) to design a procedure for computing the certified radius where the confidence is guaranteed to stay above a certain threshold. Our experimental results on CIFAR-10 and ImageNet datasets show that using information about the distribution of the confidence scores allows us to achieve a significantly better certified radius than ignoring it. Thus, we demonstrate that extra information about the base classifier at the input point can help improve certified guarantees for the smoothed classifier. Code for the experiments is available at https://github.com/aounon/cdf-smoothing.

References in corpus (15)

• End to End Learning for Self-Driving Cars
• On Calibration of Modern Neural Networks
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Adversarial Robustness via Randomized Smoothing
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• On the (Statistical) Detection of Adversarial Examples
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Semidefinite relaxations for certifying robustness to adversarial examples
• Certified Adversarial Robustness with Additive Noise
• Functional Adversarial Attacks
• Certified Defenses for Adversarial Patches
• (De)Randomized Smoothing for Certifiable Defense against Patch Attacks
• Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers
• Curse of Dimensionality on Randomized Smoothing for Certifiable Robustness

Cited by in corpus (1)

• Multi-head Ensemble of Smoothed Classifiers for Certified Robustness