353 citations · 1.2k across the 32 of their papers we have counts for
Showing 2019Show all
3 papers · 1 filter
stat.ML2019★ 3 cited
Predicting with High Correlation Features
Devansh Arpit, Caiming Xiong, Richard Socher
It has been shown that instead of learning actual object features, deep networks tend to exploit non-robust (spurious) discriminative features that are shared between training and…
stat.ML2019
How to Initialize your Network? Robust Initialization for WeightNorm & ResNets
Devansh Arpit, Victor Campos, Yoshua Bengio
Residual networks (ResNet) and weight normalization play an important role in various deep learning applications. However, parameter initialization strategies have not been studied…
stat.ML2019★ 14 cited
The Benefits of Over-parameterization at Initialization in Deep ReLU Networks
Devansh Arpit, Yoshua Bengio
It has been noted in existing literature that over-parameterization in ReLU networks generally improves performance. While there could be several factors involved behind this, we p…