20 citations · 48 across the 8 of their papers we have counts for
8 papers
Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradients
Katharina Flügel, Daniel Coquelin, Marie Weiel +3
The gradients used to train neural networks are typically computed using backpropagation. While an efficient way to obtain exact gradients, backpropagation is computationally expen…
AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning
Daniel Coquelin, Katherina Flügel, Marie Weiel +5
Communication bottlenecks severely hinder the scalability of distributed neural network training, particularly in high-performance computing (HPC) environments. We introduce AB-tra…
Harnessing Orthogonality to Train Low-Rank Neural Networks
Daniel Coquelin, Katharina Flügel, Marie Weiel +4
This study explores the learning dynamics of neural networks by analyzing the singular value decomposition (SVD) of their weights throughout training. Our investigation reveals tha…
Feed-Forward Optimization With Delayed Feedback for Neural Network Training
Katharina Flügel, Daniel Coquelin, Marie Weiel +3
Backpropagation has long been criticized for being biologically implausible due to its reliance on concepts that are not viable in natural learning processes. Two core issues are t…
Massively Parallel Genetic Optimization through Asynchronous Propagation of Populations
Oskar Taubert, Marie Weiel, Daniel Coquelin +5
We present Propulate, an evolutionary optimization algorithm and software package for global optimization and in particular hyperparameter search. For efficient use of HPC resource…
HyDe: The First Open-Source, Python-Based, GPU-Accelerated Hyperspectral Denoising Package
Daniel Coquelin, Behnood Rasti, Markus Götz +3
As with any physical instrument, hyperspectral cameras induce different kinds of noise in the acquired data. Therefore, Hyperspectral denoising is a crucial step for analyzing hype…
Accelerating Neural Network Training with Distributed Asynchronous and Selective Optimization (DASO)
Daniel Coquelin, Charlotte Debus, Markus Götz +4
With increasing data and model complexities, the time required to train neural networks has become prohibitively large. To address the exponential rise in training time, users are…
HeAT -- a Distributed and GPU-accelerated Tensor Framework for Data Analytics
Markus Götz, Daniel Coquelin, Charlotte Debus +9
To cope with the rapid growth in available data, the efficiency of data analysis and machine learning libraries has recently received increased attention. Although great advancemen…