Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025
Beyond Gaussian Initializations: Signal Preserving Weight Initialization for Odd-Sigmoid Activations
Hyunwoo Lee, Hayoung Choi, Hyunju Kim
Activation functions critically influence trainability and expressivity, and recent work has therefore explored a broad range of nonlinearities. However, widely used Gaussian i.i.d…
cs.LG2024
Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis
Hyunwoo Lee, Hayoung Choi, Hyunju Kim
As a neural network's depth increases, it can improve generalization performance. However, training deep networks is challenging due to gradient and signal propagation issues. To a…
cs.LG2023
Improved weight initialization for deep and narrow feedforward neural network
Hyunwoo Lee, Yunho Kim, Seung Yeop Yang +1
Appropriate weight initialization settings, along with the ReLU activation function, have become cornerstones of modern deep learning, enabling the training and deployment of highl…