5 papers · 1 filter
AutoPQ: Automating Quantile estimation from Point forecasts in the context of sustainability
Stefan Meisenbacher, Kaleb Phipps, Oskar Taubert +4
Optimizing smart grid operations relies on critical decision-making informed by uncertainty quantification, making probabilistic forecasting a vital tool. Designing such forecastin…
Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradients
Katharina Flügel, Daniel Coquelin, Marie Weiel +3
The gradients used to train neural networks are typically computed using backpropagation. While an efficient way to obtain exact gradients, backpropagation is computationally expen…
AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning
Daniel Coquelin, Katherina Flügel, Marie Weiel +5
Communication bottlenecks severely hinder the scalability of distributed neural network training, particularly in high-performance computing (HPC) environments. We introduce AB-tra…
Harnessing Orthogonality to Train Low-Rank Neural Networks
Daniel Coquelin, Katharina Flügel, Marie Weiel +4
This study explores the learning dynamics of neural networks by analyzing the singular value decomposition (SVD) of their weights throughout training. Our investigation reveals tha…
Feed-Forward Optimization With Delayed Feedback for Neural Network Training
Katharina Flügel, Daniel Coquelin, Marie Weiel +3
Backpropagation has long been criticized for being biologically implausible due to its reliance on concepts that are not viable in natural learning processes. Two core issues are t…