2 papers
cs.LG2025
Beyond Gaussian Initializations: Signal Preserving Weight Initialization for Odd-Sigmoid Activations
Hyunwoo Lee, Hayoung Choi, Hyunju Kim
Activation functions critically influence trainability and expressivity, and recent work has therefore explored a broad range of nonlinearities. However, widely used Gaussian i.i.d…
cs.LG2025
Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis
Hyunwoo Lee, Hayoung Choi, Hyunju Kim
As a neural network's depth increases, it can improve generalization performance. However, training deep networks is challenging due to gradient and signal propagation issues. To a…