2 citations · 3 across the 7 of their papers we have counts for
4 papers · 1 filter
Magnitude and Angle Dynamics in Training Single ReLU Neurons
Sangmin Lee, Byeongsu Sim, Jong Chul Ye
To understand learning the dynamics of deep ReLU networks, we investigate the dynamic system of gradient flow by decomposing it to magnitude and angle $ϕ(t):= π- θ(t)…
Self-supervised debiasing using low rank regularization
Geon Yeong Park, Chanyong Jung, Sangmin Lee +2
Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability. While most existing debiasing methods require full supervision on either sp…
Training Debiased Subnetworks with Contrastive Weight Pruning
Geon Yeong Park, Sangmin Lee, Sang Wan Lee +1
Neural networks are often biased to spuriously correlated features that provide misleading statistical evidence that does not generalize. This raises an interesting question: ``Doe…
Support Vectors and Gradient Dynamics of Single-Neuron ReLU Networks
Sangmin Lee, Byeongsu Sim, Jong Chul Ye
Understanding implicit bias of gradient descent for generalization capability of ReLU networks has been an important research topic in machine learning research. Unfortunately, eve…