6 citations · 19 across the 13 of their papers we have counts for
3 papers · 1 filter
A geometric framework for momentum-based optimizers for low-rank training
Steffen Schotthöfer, Timon Klein, Jonas Kusch
Low-rank pre-training and fine-tuning have recently emerged as promising techniques for reducing the computational and storage costs of large neural networks. Training low-rank par…
Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks
Steffen Schotthöfer, H. Lexie Yang, Stefan Schnake
Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness…
An Augmented Backward-Corrected Projector Splitting Integrator for Dynamical Low-Rank Training
Jonas Kusch, Steffen Schotthöfer, Alexandra Walter
Layer factorization has emerged as a widely used technique for training memory-efficient neural networks. However, layer factorization methods face several challenges, particularly…