1 citations · 1 across the 9 of their papers we have counts for
10 papers · 1 filter
From Geometry to Generalization: Why Row Normalization Can Beat Adam and Muon
Jihwan Kim, Dogyoon Song, Chulhee Yun
Different optimizers can fit the same training data while selecting classifiers with substantially different geometries, but whether this difference provably affects population per…
Implicit Bias and Loss of Plasticity in Matrix Completion: Depth Promotes Low-Rankness
Baekrok Shin, Chulhee Yun
We study matrix completion via deep matrix factorization (a.k.a. deep linear neural networks) as a simplified testbed to examine how network depth influences training dynamics. Des…
Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?
Jihwan Kim, Dogyoon Song, Chulhee Yun
We study scaling laws of signSGD under a power-law random features (PLRF) model that accounts for both feature and target decay. We analyze the population risk of a linear model tr…
Minor First, Major Last: A Depth-Induced Implicit Bias of Sharpness-Aware Minimization
Chaewon Moon, Dongkuk Si, Chulhee Yun
We study the implicit bias of Sharpness-Aware Minimization (SAM) when training -layer linear diagonal networks on linearly separable binary classification. For linear models ($L…
Uniform Spectral Growth and Convergence of Muon in LoRA-Style Matrix Factorization
Changmin Kang, Jihun Yun, Baekrok Shin +2
Spectral gradient descent (SpecGD) orthogonalizes the matrix parameter updates and has inspired practical optimizers such as Muon. They often perform well in large language model (…
From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning
Junsoo Oh, Jerry Song, Chulhee Yun
Weak-to-strong generalization refers to the phenomenon where a stronger model trained under supervision from a weaker one can outperform its teacher. While prior studies aim to exp…
Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertainty
Yeseul Cho, Baekrok Shin, Changmin Kang +1
Recent advances in deep learning rely heavily on massive datasets, leading to substantial storage and training costs. Dataset pruning aims to alleviate this demand by discarding re…
DASH: Warm-Starting Neural Network Training in Stationary Settings without Loss of Plasticity
Baekrok Shin, Junsoo Oh, Hanseul Cho +1
Warm-starting neural network training by initializing networks with previously learned weights is appealing, as practical neural networks are often deployed under a continuous infl…
Provable Benefit of Cutout and CutMix for Feature Learning
Junsoo Oh, Chulhee Yun
Patch-level data augmentation techniques such as Cutout and CutMix have demonstrated significant efficacy in enhancing the performance of vision tasks. However, a comprehensive the…
Provable Benefit of Mixup for Finding Optimal Decision Boundaries
Junsoo Oh, Chulhee Yun
We investigate how pair-wise data augmentation techniques like Mixup affect the sample complexity of finding optimal decision boundaries in a binary linear classification problem.…