Foolbox: A Python toolbox to benchmark the robustness of machine learning models

Abstract

Even todays most advanced machine learning models are easily fooled by almost imperceptible perturbations of their inputs. Foolbox is a new Python package to generate such adversarial perturbations and to quantify and compare the robustness of machine learning models. It is build around the idea that the most comparable robustness measure is the minimum perturbation needed to craft an adversarial example. To this end, Foolbox provides reference implementations of most published adversarial attack methods alongside some new ones, all of which perform internal hyperparameter tuning to find the minimum adversarial perturbation. Additionally, Foolbox interfaces with most popular deep learning frameworks such as PyTorch, Keras, TensorFlow, Theano and MXNet and allows different adversarial criteria such as targeted misclassification and top-k misclassification as well as different distance measures. The code is licensed under the MIT license and is openly available at https://github.com/bethgelab/foolbox . The most up-to-date documentation can be found at http://foolbox.readthedocs.io .

arXiv commentCode and examples available at https://github.com/bethgelab/foolbox and documentation available at http://foolbox.readthedocs.io

Cited by in corpus (15)

• OpenAttack: An Open-source Textual Adversarial Attack Toolkit
• PatchUp: A Feature-Space Block-Level Regularization Technique for Convolutional Neural Networks
• Understanding Real-world Threats to Deep Learning Models in Android Apps
• SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networks
• Assaying Out-Of-Distribution Generalization in Transfer Learning
• Task dependent Deep LDA pruning of neural networks
• Mutual Adversarial Training: Learning together is better than going alone
• Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics
• Adversarial Defense by Latent Style Transformations
• Imperceptible Adversarial Examples by Spatial Chroma-Shift
• secml: A Python Library for Secure and Explainable Machine Learning
• Denoised Internal Models: a Brain-Inspired Autoencoder against Adversarial Attacks
• Guiding the retraining of convolutional neural networks against adversarial inputs
• Neural Belief Reasoner
• Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Study