6 papers
SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning
Guangyuan Wang, Mads Toftrup, Sebastian Loeschcke +2
Physics-informed neural networks (PINNs) often face ill-conditioned objectives that limit high-accuracy training. Dense quasi-Newton methods improve local conditioning but require…
M+Adam: Low-Precision Training via Additive-Multiplicative Optimization
Xiaoyuan Liang, Sebastian Loeschcke, Mads Toftrup +1
Training with quantized weights can reduce costs but often results in degraded accuracy, especially when optimization is carried out in low precision, without storing high-precisio…
LoQT: Low-Rank Adapters for Quantized Pretraining
Sebastian Loeschcke, Mads Toftrup, Michael J. Kastoryano +2
Despite advances using low-rank adapters and quantization, pretraining of large models on consumer hardware has not been possible without model sharding, offloading during training…
On Generalization Bounds for Projective Clustering
Maria Sofia Bucarelli, Matilde Fjeldsø Larsen, Chris Schwiegelshohn +1
Given a set of points, clustering consists of finding a partition of a point set into clusters such that the center to which a point is assigned is as close as possible. Most c…
The Power of Uniform Sampling for Coresets
Vladimir Braverman, Vincent Cohen-Addad, Shaofeng H. -C. Jiang +4
Motivated by practical generalizations of the classic -median and -means objectives, such as clustering with size constraints, fair clustering, and Wasserstein barycenter, we…
A reproduction of Apple's bi-directional LSTM models for language identification in short strings
Mads Toftrup, Søren Asger Sørensen, Manuel R. Ciosici +1
Language Identification is the task of identifying a document's language. For applications like automatic spell checker selection, language identification must use very short strin…