2 citations · 6 across the 15 of their papers we have counts for
12 papers · 1 filter
Rethinking Likelihood distributions: Student's t Likelihood Boosts Bayesian Neural Network Performance
Pei-Hsuan Hsia, Lars H. Heyen, Arvid Weyrauch +4
In Bayesian neural networks (BNNs), variational inference is a widely adopted framework for modeling uncertainty in a distributional way, with the evidence lower bound (ELBO) servi…
Sampling Parallelism for Fast and Efficient Bayesian Learning
Asena Karolin Özdemir, Lars H. Heyen, Arvid Weyrauch +3
Machine learning models, and deep neural networks in particular, are increasingly deployed in risk-sensitive domains such as healthcare, environmental forecasting, and finance, whe…
Bayesian Lottery Ticket Hypothesis
Nicholas Kuhn, Arvid Weyrauch, Lars Heyen +3
Bayesian neural networks (BNNs) are a useful tool for uncertainty quantification, but require substantially more computational resources than conventional neural networks. For non-…
Energy Consumption in Parallel Neural Network Training
Philipp Huber, David Li, Juan Pedro Gutiérrez Hermosillo Muriedas +4
The increasing demand for computational resources of training neural networks leads to a concerning growth in energy consumption. While parallelization has enabled upscaling model…
Jigsaw: Training Multi-Billion-Parameter AI Weather Models with Optimized Model Parallelism
Deifilia Kieckhefen, Markus Götz, Lars H. Heyen +2
AI-based methods have revolutionized atmospheric forecasting, with recent successes in medium-range forecasting spurring the development of climate foundation models. Accurate mode…
PETNet -- Coincident Particle Event Detection using Spiking Neural Networks
Jan Debus, Charlotte Debus, Günther Dissertori +1
Spiking neural networks (SNN) hold the promise of being a more biologically plausible, low-energy alternative to conventional artificial neural networks. Their time-variant nature…
Model Fusion via Neuron Transplantation
Muhammed Öz, Nicholas Kiefer, Charlotte Debus +3
Ensemble learning is a widespread technique to improve the prediction performance of neural networks. However, it comes at the price of increased memory and inference time. In this…
A Comparative Study of Pruning Methods in Transformer-based Time Series Forecasting
Nicholas Kiefer, Arvid Weyrauch, Muhammed Öz +3
The current landscape in time-series forecasting is dominated by Transformer-based models. Their high parameter count and corresponding demand in computational resources pose a cha…
AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning
Daniel Coquelin, Katherina Flügel, Marie Weiel +5
Communication bottlenecks severely hinder the scalability of distributed neural network training, particularly in high-performance computing (HPC) environments. We introduce AB-tra…
ReCycle: Fast and Efficient Long Time Series Forecasting with Residual Cyclic Transformers
Arvid Weyrauch, Thomas Steens, Oskar Taubert +6
Transformers have recently gained prominence in long time series forecasting by elevating accuracies in a variety of use cases. Regrettably, in the race for better predictive perfo…
Harnessing Orthogonality to Train Low-Rank Neural Networks
Daniel Coquelin, Katharina Flügel, Marie Weiel +4
This study explores the learning dynamics of neural networks by analyzing the singular value decomposition (SVD) of their weights throughout training. Our investigation reveals tha…
Feed-Forward Optimization With Delayed Feedback for Neural Network Training
Katharina Flügel, Daniel Coquelin, Marie Weiel +3
Backpropagation has long been criticized for being biologically implausible due to its reliance on concepts that are not viable in natural learning processes. Two core issues are t…