From the 1 of 6 linked papers with an AI index.
6 papers
AMUSE: Anytime Muon with Stable Gradient Evaluation
Jueun Kim, Baekrok Shin, Jihun Yun +3
The paper introduces AMUSE, an optimization method that combines Muon's orthogonal momentum updates with Schedule-Free iterate averaging to achieve fast, stable training without le…
Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling
Yeseul Cho, Baekrok Shin, Changmin Kang +1
Dataset pruning reduces the storage and training costs of deep learning by selecting an informative subset from a large dataset. However, most existing pruning methods require full…
Implicit Bias and Loss of Plasticity in Matrix Completion: Depth Promotes Low-Rankness
Baekrok Shin, Chulhee Yun
We study matrix completion via deep matrix factorization (a.k.a. deep linear neural networks) as a simplified testbed to examine how network depth influences training dynamics. Des…
Uniform Spectral Growth and Convergence of Muon in LoRA-Style Matrix Factorization
Changmin Kang, Jihun Yun, Baekrok Shin +2
Spectral gradient descent (SpecGD) orthogonalizes the matrix parameter updates and has inspired practical optimizers such as Muon. They often perform well in large language model (…
Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertainty
Yeseul Cho, Baekrok Shin, Changmin Kang +1
Recent advances in deep learning rely heavily on massive datasets, leading to substantial storage and training costs. Dataset pruning aims to alleviate this demand by discarding re…
DASH: Warm-Starting Neural Network Training in Stationary Settings without Loss of Plasticity
Baekrok Shin, Junsoo Oh, Hanseul Cho +1
Warm-starting neural network training by initializing networks with previously learned weights is appealing, as practical neural networks are often deployed under a continuous infl…