282 citations · 989 across the 11 of their papers we have counts for
4 papers · 1 filter
Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions
Ameya D. Jagtap, Yeonjong Shin, Kenji Kawaguchi +1
We propose a new type of neural networks, Kronecker neural networks (KNNs), that form a general framework for neural networks with adaptive activation functions. KNNs employ the Kr…
Plateau Phenomenon in Gradient Descent Training of ReLU networks: Explanation, Quantification and Avoidance
Mark Ainsworth, Yeonjong Shin
The ability of neural networks to provide `best in class' approximation across a wide range of applications is well-documented. Nevertheless, the powerful expressivity of neural ne…
Effects of Depth, Width, and Initialization: A Convergence Analysis of Layer-wise Training for Deep Linear Neural Networks
Yeonjong Shin
Deep neural networks have been used in various machine learning applications and achieved tremendous empirical successes. However, training deep neural networks is a challenging ta…
Trainability of ReLU networks and Data-dependent Initialization
Yeonjong Shin, George Em Karniadakis
In this paper, we study the trainability of rectified linear unit (ReLU) networks. A ReLU neuron is said to be dead if it only outputs a constant for any input. Two death states of…