Inherent Weight Normalization in Stochastic Neural Networks

Georgios Detorakis, Sourav Dutta, Abhishek Khanna +3

Multiplicative stochasticity such as Dropout improves the robustness and generalizability of deep neural networks. Here, we further demonstrate that always-on multiplicative stocha…