2 papers
cs.LG2024
ZNorm: Z-Score Gradient Normalization Accelerating Skip-Connected Network Training without Architectural Modification
Juyoung Yun
The rapid advancements in deep learning necessitate better training methods for deep neural networks (DNNs). As models grow in complexity, vanishing and exploding gradients impede…
cs.LG2024
Mitigating Gradient Overlap in Deep Residual Networks with Gradient Normalization for Improved Non-Convex Optimization
Juyoung Yun
In deep learning, Residual Networks (ResNets) have proven effective in addressing the vanishing gradient problem, allowing for the successful training of very deep networks. Howeve…