3 papers
cs.LG2024
Mitigating Gradient Overlap in Deep Residual Networks with Gradient Normalization for Improved Non-Convex Optimization
Juyoung Yun
In deep learning, Residual Networks (ResNets) have proven effective in addressing the vanishing gradient problem, allowing for the successful training of very deep networks. Howeve…
cs.LG2024
ZNorm: Z-Score Gradient Normalization Accelerating Skip-Connected Network Training without Architectural Modification
Juyoung Yun
The rapid advancements in deep learning necessitate better training methods for deep neural networks (DNNs). As models grow in complexity, vanishing and exploding gradients impede…
cs.LG2023
Linear Oscillation: A Novel Activation Function for Vision Transformer
Juyoung Yun
Activation functions are the linchpins of deep learning, profoundly influencing both the representational capacity and training dynamics of neural networks. They shape not only the…