1 citations · 3 across the 4 of their papers we have counts for
4 papers · 1 filter
Model Fusion via Neuron Transplantation
Muhammed Öz, Nicholas Kiefer, Charlotte Debus +3
Ensemble learning is a widespread technique to improve the prediction performance of neural networks. However, it comes at the price of increased memory and inference time. In this…
A Comparative Study of Pruning Methods in Transformer-based Time Series Forecasting
Nicholas Kiefer, Arvid Weyrauch, Muhammed Öz +3
The current landscape in time-series forecasting is dominated by Transformer-based models. Their high parameter count and corresponding demand in computational resources pose a cha…
AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning
Daniel Coquelin, Katherina Flügel, Marie Weiel +5
Communication bottlenecks severely hinder the scalability of distributed neural network training, particularly in high-performance computing (HPC) environments. We introduce AB-tra…
Harnessing Orthogonality to Train Low-Rank Neural Networks
Daniel Coquelin, Katharina Flügel, Marie Weiel +4
This study explores the learning dynamics of neural networks by analyzing the singular value decomposition (SVD) of their weights throughout training. Our investigation reveals tha…