11 citations · 26 across the 18 of their papers we have counts for
18 papers
Rethinking Likelihood distributions: Student's t Likelihood Boosts Bayesian Neural Network Performance
Pei-Hsuan Hsia, Lars H. Heyen, Arvid Weyrauch +4
In Bayesian neural networks (BNNs), variational inference is a widely adopted framework for modeling uncertainty in a distributional way, with the evidence lower bound (ELBO) servi…
Sampling Parallelism for Fast and Efficient Bayesian Learning
Asena Karolin Özdemir, Lars H. Heyen, Arvid Weyrauch +3
Machine learning models, and deep neural networks in particular, are increasingly deployed in risk-sensitive domains such as healthcare, environmental forecasting, and finance, whe…
Inverse Design of Optical Multilayer Thin Films using Robust Masked Diffusion Models
Jonas Schaible, Asena Karolin Özdemir, Charlotte Debus +5
Inverse design of optical multilayer stacks seeks to infer layer materials, thicknesses, and ordering from a desired target spectrum. It is a long-standing challenge due to the lar…
Differentiable Power-Flow Optimization
Muhammed Öz, Jasmin Hörter, Kaleb Phipps +3
With the rise of renewable energy sources and their high variability in generation, the management of power grids becomes increasingly complex and computationally demanding. Conven…
Bayesian Lottery Ticket Hypothesis
Nicholas Kuhn, Arvid Weyrauch, Lars Heyen +3
Bayesian neural networks (BNNs) are a useful tool for uncertainty quantification, but require substantially more computational resources than conventional neural networks. For non-…
pyGinkgo: A Sparse Linear Algebra Operator Framework for Python
Keshvi Tuteja, Gregor Olenik, Roman Mishchuk +5
Sparse linear algebra is a cornerstone of many scientific computing and machine learning applications. Python has become a popular choice for these applications due to its simplici…
Energy Consumption in Parallel Neural Network Training
Philipp Huber, David Li, Juan Pedro Gutiérrez Hermosillo Muriedas +4
The increasing demand for computational resources of training neural networks leads to a concerning growth in energy consumption. While parallelization has enabled upscaling model…
Jigsaw: Training Multi-Billion-Parameter AI Weather Models with Optimized Model Parallelism
Deifilia Kieckhefen, Markus Götz, Lars H. Heyen +2
AI-based methods have revolutionized atmospheric forecasting, with recent successes in medium-range forecasting spurring the development of climate foundation models. Accurate mode…
Model Fusion via Neuron Transplantation
Muhammed Öz, Nicholas Kiefer, Charlotte Debus +3
Ensemble learning is a widespread technique to improve the prediction performance of neural networks. However, it comes at the price of increased memory and inference time. In this…
A Comparative Study of Pruning Methods in Transformer-based Time Series Forecasting
Nicholas Kiefer, Arvid Weyrauch, Muhammed Öz +3
The current landscape in time-series forecasting is dominated by Transformer-based models. Their high parameter count and corresponding demand in computational resources pose a cha…
Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradients
Katharina Flügel, Daniel Coquelin, Marie Weiel +3
The gradients used to train neural networks are typically computed using backpropagation. While an efficient way to obtain exact gradients, backpropagation is computationally expen…
AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning
Daniel Coquelin, Katherina Flügel, Marie Weiel +5
Communication bottlenecks severely hinder the scalability of distributed neural network training, particularly in high-performance computing (HPC) environments. We introduce AB-tra…
ReCycle: Fast and Efficient Long Time Series Forecasting with Residual Cyclic Transformers
Arvid Weyrauch, Thomas Steens, Oskar Taubert +6
Transformers have recently gained prominence in long time series forecasting by elevating accuracies in a variety of use cases. Regrettably, in the race for better predictive perfo…
Harnessing Orthogonality to Train Low-Rank Neural Networks
Daniel Coquelin, Katharina Flügel, Marie Weiel +4
This study explores the learning dynamics of neural networks by analyzing the singular value decomposition (SVD) of their weights throughout training. Our investigation reveals tha…
Feed-Forward Optimization With Delayed Feedback for Neural Network Training
Katharina Flügel, Daniel Coquelin, Marie Weiel +3
Backpropagation has long been criticized for being biologically implausible due to its reliance on concepts that are not viable in natural learning processes. Two core issues are t…
Massively Parallel Genetic Optimization through Asynchronous Propagation of Populations
Oskar Taubert, Marie Weiel, Daniel Coquelin +5
We present Propulate, an evolutionary optimization algorithm and software package for global optimization and in particular hyperparameter search. For efficient use of HPC resource…
HyDe: The First Open-Source, Python-Based, GPU-Accelerated Hyperspectral Denoising Package
Daniel Coquelin, Behnood Rasti, Markus Götz +3
As with any physical instrument, hyperspectral cameras induce different kinds of noise in the acquired data. Therefore, Hyperspectral denoising is a crucial step for analyzing hype…
Energy-Aware Cloud Management through Progressive SLA Specification
Dražen Lučanin, Foued Jrad, Ivona Brandic +1
Novel energy-aware cloud management methods dynamically reallocate computation across geographically distributed data centers to leverage regional electricity price and temperature…