353 citations · 1.2k across the 32 of their papers we have counts for
15 papers · 1 filter
TPV: Parameter Perturbations Through the Lens of Test Prediction Variance
Devansh Arpit
We introduce test prediction variance (TPV)--the first-order sensitivity of a trained model's outputs to parameter perturbations--as a unifying framework for analyzing post-trainin…
Neural Bayes: A Generic Parameterization Method for Unsupervised Representation Learning
Devansh Arpit, Huan Wang, Caiming Xiong +2
We introduce a parameterization method called Neural Bayes which allows computing statistical quantities that are in general difficult to compute and opens avenues for formulating…
Predicting with High Correlation Features
Devansh Arpit, Caiming Xiong, Richard Socher
It has been shown that instead of learning actual object features, deep networks tend to exploit non-robust (spurious) discriminative features that are shared between training and…
How to Initialize your Network? Robust Initialization for WeightNorm & ResNets
Devansh Arpit, Victor Campos, Yoshua Bengio
Residual networks (ResNet) and weight normalization play an important role in various deep learning applications. However, parameter initialization strategies have not been studied…
The Benefits of Over-parameterization at Initialization in Deep ReLU Networks
Devansh Arpit, Yoshua Bengio
It has been noted in existing literature that over-parameterization in ReLU networks generally improves performance. While there could be several factors involved behind this, we p…
h-detach: Modifying the LSTM Gradient Towards Better Optimization
Devansh Arpit, Bhargav Kanuparthi, Giancarlo Kerg +3
Recurrent neural networks are known for their notorious exploding and vanishing gradient problem (EVGP). This problem becomes more evident in tasks where the information needed to…