4 papers · 1 filter
Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence
Tien-Phat Nguyen, Truong Nguyen, Minh-Phuc Truong +3
Muon orthogonalizes the momentum buffer before each update, replacing its singular values with ones via Newton-Schulz iterations. This simple change lets Muon tolerate far larger l…
On the Use of Bagging for Local Intrinsic Dimensionality Estimation
Kristóf Péter, Ricardo J. G. B. Campello, James Bailey +1
The theory of Local Intrinsic Dimensionality (LID) has become a valuable tool for characterizing local complexity within and across data manifolds, supporting a range of data minin…
LDReg: Local Dimensionality Regularized Self-Supervised Learning
Hanxun Huang, Ricardo J. G. B. Campello, Sarah Monazam Erfani +3
Representations learned via self-supervised learning (SSL) can be susceptible to dimensional collapse, where the learned representation subspace is of extremely low dimensionality…
Dimensionality-Aware Outlier Detection: Theoretical and Experimental Analysis
Alastair Anderberg, James Bailey, Ricardo J. G. B. Campello +4
We present a nonparametric method for outlier detection that takes full account of local variations in intrinsic dimensionality within the dataset. Using the theory of Local Intrin…