Tensor Normalization and Full Distribution Training
arXiv:2109.02345
Abstract
In this work, we introduce pixel wise tensor normalization, which is inserted after rectifier linear units and, together with batch normalization, provides a significant improvement in the accuracy of modern deep neural networks. In addition, this work deals with the robustness of networks. We show that the factorized superposition of images from the training set and the reformulation of the multi class problem into a multi-label problem yields significantly more robust networks. The reformulation and the adjustment of the multi class log loss also improves the results compared to the overlay with only one class as label. https://atreus.informatik.uni-tuebingen.de/seafile/d/8e2ab8c3fdd444e1a135/?p=%2FTNandFDT&mode=list
References in corpus (20)
- Adam: A Method for Stochastic Optimization
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- On the difficulty of training Recurrent Neural Networks
- Instance Normalization: The Missing Ingredient for Fast Stylization
- Wide Residual Networks
- Simultaneous Feature Learning and Hash Coding with Deep Neural Networks
- CNN: Single-label to Multi-label
- Deep Convolutional Ranking for Multilabel Image Annotation
- Improving Adversarial Robustness via Promoting Ensemble Diversity
- PupilNet: Convolutional Neural Networks for Robust Pupil Detection
- TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types
- Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks
- Discretization based Solutions for Secure Machine Learning against Adversarial Attacks
- Noisy multi-label semi-supervised dimensionality reduction
- Fast camera focus estimation for gaze-based focus control
- Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functions
- Top-k Multiclass SVM
- Eye movement simulation and detector creation to reduce laborious parameter adjustments
- Multi Layer Neural Networks as Replacement for Pooling Operations