57 citations · 131 across the 64 of their papers we have counts for
11 papers · 1 filter
Inner Momentum for Differentially Private Muon
Bishnu Bhusal, Minh Vu, Ben Southworth +3
Differentially private training clips each per-example gradient before adding noise. This clipping is radial for each example, yet unequal clipping factors can distort the relative…
An Efficient Solver for Finite Element-based Constrained Transport in 3D Magnetohydrodynamics Applied to Magnetic Confinement Fusion
Golo A. Wimmer, Konstantin Lipnikov, Ben S. Southworth +1
We present an efficient solver framework for the stiff magnetic wave coupling arising in resistive magnetohydrodynamics (MHD) on realistic tokamak geometries. The approach builds o…
Shock solutions for the one-dimensional information geometric regularization of compressible flow
William Barham, Brian K. Tran, Ben S. Southworth +1
The information geometric regularization (IGR) is an inviscid regularization of the compressible Euler equations that alters the geometry of Lagrangian characteristics to prevent t…
Broken-space Additive Schwarz Mass Inverse Approximations and (Block) Preconditioning
Oliver A. Krzysik, Ben S. Southworth, Golo A. Wimmer
Finite-element mass matrix solves and approximate inverses arise often in explicit time integration as well as Schur-complement-based block preconditioning. A diagonal approximatio…
A black-box, multilevel algebraic preconditioning framework for conforming finite elements
O. A. Krzysik, B. S. Southworth, G. A. Wimmer
Recently we introduced the least-squares algebraic-multigrid domain-decomposition (LS-AMG-DD) method as a multilevel, algebraic preconditioner for sparse symmetric positive definit…
Two-level convergence of Algebraic Multigrid with Overlapping Smoothers and Spectral Coarse Grids
O. A. Krzysik, B. S. Southworth, H. Al Daas
We recently developed the least-squares algebraic-multigrid domain-decomposition (LS-AMG-DD) solver as an algebraic multilevel method for sparse symmetric positive definite matrice…
Muon in Vision Transformers: Optimizer-Recipe Interactions and Gradient Spectra
Ben S. Southworth, Shuai Jiang, Daniel McBride +2
Muon is a recently developed matrix-aware optimizer that has shown strong results in transformer training, but its behavior in vision transformers (ViTs) is not yet well understood…
Beyond Muon: MUD (MomentUm Decorrelation) for Faster Transformer Training
Ben S. Southworth, Stephen Thomas
Orthogonalized-momentum optimizers such as Muon improve transformer training by approximately whitening/orthogonalizing matrix-valued momentum updates via a short polar-decompositi…
Multilevel Training for Kolmogorov Arnold Networks
Ben S. Southworth, Jonas A. Actor, Graham Harper +1
Algorithmic speedup of training common neural architectures is made difficult by the lack of structure guaranteed by the function compositions inherent to such networks. In contras…
Algebraic Multigrid with Overlapping Schwarz Smoothers and Local Spectral Coarse Grids for Least Squares Problems
Ben S. Southworth, Hussam Al Daas, Golo A. Wimmer +1
This paper develops a new algebraic multigrid (AMG) method for sparse least-squares systems of the form motivated by challenging applications in scientific computing where…
On the Convergence Behavior of Preconditioned Gradient Descent Toward the Rich Learning Regime
Shuai Jiang, Alexey Voronin, Eric Cyr +1
Spectral bias, the tendency of neural networks to learn low frequencies first, can be both a blessing and a curse. While it enhances the generalization capabilities by suppressing…