107 citations · 146 across the 6 of their papers we have counts for
Showing 2020Show all
2 papers · 1 filter
cs.LG2020
Neural Path Features and Neural Path Kernel : Understanding the role of gates in deep learning
Chandrashekar Lakshminarayanan, Amit Vikram Singh
Rectified linear unit (ReLU) activations can also be thought of as 'gates', which, either pass or stop their pre-activation input when they are 'on' (when the pre-activation input…
cs.LG2020
Deep Gated Networks: A framework to understand training and generalisation in deep learning
Chandrashekar Lakshminarayanan, Amit Vikram Singh
Understanding the role of (stochastic) gradient descent (SGD) in the training and generalisation of deep neural networks (DNNs) with ReLU activation has been the object study in th…