Tensor Normalization and Full Distribution Training
arXiv:2109.02345
Abstract
In this work, we introduce pixel wise tensor normalization, which is inserted after rectifier linear units and, together with batch normalization, provides a significant improvement in the accuracy of modern deep neural networks. In addition, this work deals with the robustness of networks. We show that the factorized superposition of images from the training set and the reformulation of the multi class problem into a multi-label problem yields significantly more robust networks. The reformulation and the adjustment of the multi class log loss also improves the results compared to the overlay with only one class as label. https://atreus.informatik.uni-tuebingen.de/seafile/d/8e2ab8c3fdd444e1a135/?p=%2FTNandFDT&mode=list
References in corpus (6)
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- On the difficulty of training Recurrent Neural Networks
- Improving Adversarial Robustness via Promoting Ensemble Diversity
- Discretization based Solutions for Secure Machine Learning against Adversarial Attacks
- Noisy multi-label semi-supervised dimensionality reduction
- Fast camera focus estimation for gaze-based focus control