Robust Deep Neural Networks Inspired by Fuzzy Logic

Abstract

Deep neural networks have achieved impressive performance and become the de-facto standard in many tasks. However, troubling phenomena such as adversarial and fooling examples suggest that the generalization they make is flawed. I argue that among the roots of the phenomena are two geometric properties of common deep learning architectures: their distributed nature and the connectedness of their decision regions. As a remedy, I propose new architectures inspired by fuzzy logic that combine several alternative design elements. Through experiments on MNIST and CIFAR-10, the new models are shown to be more local, better at rejecting noise samples, and more robust against adversarial examples. Ablation analyses reveal behaviors on adversarial examples that cannot be explained by the linearity hypothesis but are consistent with the hypothesis that logic-inspired traits create more robust models.

arXiv comment7 pages, 4 figures, source code: https://bitbucket.org/minhlab/newlogic

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Intriguing properties of neural networks
• On the Number of Linear Regions of Deep Neural Networks
• Adversarially Robust Generalization Requires More Data
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Adversarial Machine Learning at Scale
• Robustness May Be at Odds with Accuracy
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Adversarial examples from computational constraints
• Dimensions of Neural-symbolic Integration - A Structured Survey
• Adversarial Training Versus Weight Decay
• Understanding Adversarial Robustness: The Trade-off between Minimum and Average Margin
• Neural Networks with Structural Resistance to Adversarial Attacks