4 papers · 1 filter
SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning
Guangyuan Wang, Mads Toftrup, Sebastian Loeschcke +2
Physics-informed neural networks (PINNs) often face ill-conditioned objectives that limit high-accuracy training. Dense quasi-Newton methods improve local conditioning but require…
M+Adam: Low-Precision Training via Additive-Multiplicative Optimization
Xiaoyuan Liang, Sebastian Loeschcke, Mads Toftrup +1
Training with quantized weights can reduce costs but often results in degraded accuracy, especially when optimization is carried out in low precision, without storing high-precisio…
LoQT: Low-Rank Adapters for Quantized Pretraining
Sebastian Loeschcke, Mads Toftrup, Michael J. Kastoryano +2
Despite advances using low-rank adapters and quantization, pretraining of large models on consumer hardware has not been possible without model sharding, offloading during training…
On Generalization Bounds for Projective Clustering
Maria Sofia Bucarelli, Matilde Fjeldsø Larsen, Chris Schwiegelshohn +1
Given a set of points, clustering consists of finding a partition of a point set into clusters such that the center to which a point is assigned is as close as possible. Most c…