2 citations · 6 across the 15 of their papers we have counts for
16 papers
4D Parallelism Unlocks Exascale Bayesian Neural Networks for High-Fidelity Atmospheric Modeling
Deifilia Kieckhefen, Juan Pedro Gutiérrez Hermosillo Muriedas, Lars Helge Heyen +12
We present BEAST, the first-ever Bayesian Swin Transformer for atmospheric forecasting on 0.25 global resolution able to accurately quantify both aleatoric and epistemic un…
Rethinking Likelihood distributions: Student's t Likelihood Boosts Bayesian Neural Network Performance
Pei-Hsuan Hsia, Lars H. Heyen, Arvid Weyrauch +4
In Bayesian neural networks (BNNs), variational inference is a widely adopted framework for modeling uncertainty in a distributional way, with the evidence lower bound (ELBO) servi…
Sampling Parallelism for Fast and Efficient Bayesian Learning
Asena Karolin Özdemir, Lars H. Heyen, Arvid Weyrauch +3
Machine learning models, and deep neural networks in particular, are increasingly deployed in risk-sensitive domains such as healthcare, environmental forecasting, and finance, whe…
Inverse Design of Optical Multilayer Thin Films using Robust Masked Diffusion Models
Jonas Schaible, Asena Karolin Özdemir, Charlotte Debus +5
Inverse design of optical multilayer stacks seeks to infer layer materials, thicknesses, and ordering from a desired target spectrum. It is a long-standing challenge due to the lar…
Differentiable Power-Flow Optimization
Muhammed Öz, Jasmin Hörter, Kaleb Phipps +3
With the rise of renewable energy sources and their high variability in generation, the management of power grids becomes increasingly complex and computationally demanding. Conven…
Bayesian Lottery Ticket Hypothesis
Nicholas Kuhn, Arvid Weyrauch, Lars Heyen +3
Bayesian neural networks (BNNs) are a useful tool for uncertainty quantification, but require substantially more computational resources than conventional neural networks. For non-…
pyGinkgo: A Sparse Linear Algebra Operator Framework for Python
Keshvi Tuteja, Gregor Olenik, Roman Mishchuk +5
Sparse linear algebra is a cornerstone of many scientific computing and machine learning applications. Python has become a popular choice for these applications due to its simplici…
Energy Consumption in Parallel Neural Network Training
Philipp Huber, David Li, Juan Pedro Gutiérrez Hermosillo Muriedas +4
The increasing demand for computational resources of training neural networks leads to a concerning growth in energy consumption. While parallelization has enabled upscaling model…
Jigsaw: Training Multi-Billion-Parameter AI Weather Models with Optimized Model Parallelism
Deifilia Kieckhefen, Markus Götz, Lars H. Heyen +2
AI-based methods have revolutionized atmospheric forecasting, with recent successes in medium-range forecasting spurring the development of climate foundation models. Accurate mode…
PETNet -- Coincident Particle Event Detection using Spiking Neural Networks
Jan Debus, Charlotte Debus, Günther Dissertori +1
Spiking neural networks (SNN) hold the promise of being a more biologically plausible, low-energy alternative to conventional artificial neural networks. Their time-variant nature…
Model Fusion via Neuron Transplantation
Muhammed Öz, Nicholas Kiefer, Charlotte Debus +3
Ensemble learning is a widespread technique to improve the prediction performance of neural networks. However, it comes at the price of increased memory and inference time. In this…
A Comparative Study of Pruning Methods in Transformer-based Time Series Forecasting
Nicholas Kiefer, Arvid Weyrauch, Muhammed Öz +3
The current landscape in time-series forecasting is dominated by Transformer-based models. Their high parameter count and corresponding demand in computational resources pose a cha…
AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning
Daniel Coquelin, Katherina Flügel, Marie Weiel +5
Communication bottlenecks severely hinder the scalability of distributed neural network training, particularly in high-performance computing (HPC) environments. We introduce AB-tra…
ReCycle: Fast and Efficient Long Time Series Forecasting with Residual Cyclic Transformers
Arvid Weyrauch, Thomas Steens, Oskar Taubert +6
Transformers have recently gained prominence in long time series forecasting by elevating accuracies in a variety of use cases. Regrettably, in the race for better predictive perfo…
Harnessing Orthogonality to Train Low-Rank Neural Networks
Daniel Coquelin, Katharina Flügel, Marie Weiel +4
This study explores the learning dynamics of neural networks by analyzing the singular value decomposition (SVD) of their weights throughout training. Our investigation reveals tha…
Feed-Forward Optimization With Delayed Feedback for Neural Network Training
Katharina Flügel, Daniel Coquelin, Marie Weiel +3
Backpropagation has long been criticized for being biologically implausible due to its reliance on concepts that are not viable in natural learning processes. Two core issues are t…