Unreasonable Effectiveness of Last Hidden Layer Activations for Adversarial Robustness

Omer Faruk Tuna, Ferhat Ozgur Catak, M. Taner Eskil

In standard Deep Neural Network (DNN) based classifiers, the general convention is to omit the activation function in the last (output) layer and directly apply the softmax functio…