6 citations · 19 across the 13 of their papers we have counts for
15 papers
A geometric framework for momentum-based optimizers for low-rank training
Steffen Schotthöfer, Timon Klein, Jonas Kusch
Low-rank pre-training and fine-tuning have recently emerged as promising techniques for reducing the computational and storage costs of large neural networks. Training low-rank par…
Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks
Steffen Schotthöfer, H. Lexie Yang, Stefan Schnake
Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness…
An Augmented Backward-Corrected Projector Splitting Integrator for Dynamical Low-Rank Training
Jonas Kusch, Steffen Schotthöfer, Alexandra Walter
Layer factorization has emerged as a widely used technique for training memory-efficient neural networks. However, layer factorization methods face several challenges, particularly…
Windowing Regularization Techniques for Unsteady Aerodynamic Shape Optimization
Steffen Schotthöfer, Beckett Y. Zhou, Tim Albring +1
Unsteady Aerodynamic Shape Optimization presents new challenges in terms of sensitivity analysis of time-dependent objective functions. In this work, we consider periodic unsteady…
GeoLoRA: Geometric integration for parameter efficient fine-tuning
Steffen Schotthöfer, Emanuele Zangrando, Gianluca Ceruti +2
Low-Rank Adaptation (LoRA) has become a widely used method for parameter-efficient fine-tuning of large-scale, pre-trained neural networks. However, LoRA and its extensions face se…
Federated Dynamical Low-Rank Training with Global Loss Convergence Guarantees
Steffen Schotthöfer, M. Paul Laiu
In this work, we propose a federated dynamical low-rank training (FeDLRT) scheme to reduce client compute and communication costs - two significant performance bottlenecks in horiz…
Structure-preserving neural networks for the regularized entropy-based closure of the Boltzmann moment system
Steffen Schotthöfer, M. Paul Laiu, Martin Frank +1
The main challenge of large-scale numerical simulation of radiation transport is the high memory and computation time requirements of discretization methods for kinetic equations.…
Structure-Preserving Operator Learning: Modeling the Collision Operator of Kinetic Equations
Jae Yong Lee, Steffen Schotthöfer, Tianbai Xiao +2
This work explores the application of deep operator learning principles to a problem in statistical physics. Specifically, we consider the linear kinetic equation, consisting of a…
Construction of high-order conservative basis-update and Galerkin dynamical low-rank integrators
Lukas Einkemmer, Jonas Kusch, Steffen Schotthöfer
Numerical simulations of kinetic problems can become prohibitively expensive due to their large memory requirements and computational costs. A method that has proven to successfull…
Geometry-aware training of factorized layers in tensor Tucker format
Emanuele Zangrando, Steffen Schotthöfer, Gianluca Ceruti +2
Reducing parameter redundancies in neural network architectures is crucial for achieving feasible computational and memory requirements during training and inference phases. Given…
KiT-RT: An extendable framework for radiative transfer and therapy
Jonas Kusch, Steffen Schotthöfer, Pia Stammer +2
In this paper we present KiT-RT (Kinetic Transport Solver for Radiation Therapy), an open-source C++ based framework for solving kinetic equations in radiation therapy applications…
Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations
Steffen Schotthöfer, Emanuele Zangrando, Jonas Kusch +2
Neural networks have achieved tremendous success in a large variety of applications. However, their memory footprint and computational demand can render them impractical in applica…
Predicting continuum breakdown with deep neural networks
Tianbai Xiao, Steffen Schotthöfer, Martin Frank
The multi-scale nature of gaseous flows poses tremendous difficulties for theoretical and numerical analysis. The Boltzmann equation, while possessing a wider applicability than hy…
Neural network-based, structure-preserving entropy closures for the Boltzmann moment system
Steffen Schotthöfer, Tianbai Xiao, Martin Frank +1
This work presents neural network based minimal entropy closures for the moment system of the Boltzmann equation, that preserve the inherent structure of the system of partial diff…
A structure-preserving surrogate model for the closure of the moment system of the Boltzmann equation using convex deep neural networks
Steffen Schotthöfer, Tianbai Xiao, Martin Frank +1
Direct simulation of physical processes on a kinetic level is prohibitively expensive in aerospace applications due to the extremely high dimension of the solution spaces. In this…