8 citations · 11 across the 6 of their papers we have counts for
6 papers
Activation- and Influence-Aware Ranks (AIR): Function-Preserving SVD Compression for LLMs
Nico Harder, Daniel Becking, Karsten Mueller +1
We present Activation- and Influence-Aware Ranks (AIR), an SVD-based LLM compression framework that guides each weight matrix's low-rank approximation with a backward-signal influe…
Efficient Federated Learning Tiny Language Models for Mobile Network Feature Prediction
Daniel Becking, Ingo Friese, Karsten Müller +4
In telecommunications, Autonomous Networks (ANs) automatically adjust configurations based on specific requirements (e.g., bandwidth) and available resources. These networks rely o…
Adaptive Differential Filters for Fast and Communication-Efficient Federated Learning
Daniel Becking, Heiner Kirchhoffer, Gerhard Tech +4
Federated learning (FL) scenarios inherently generate a large communication overhead by frequently transmitting neural network updates between clients and server. To minimize the c…
ECQ: Explainability-Driven Quantization for Low-Bit and Sparse DNNs
Daniel Becking, Maximilian Dreyer, Wojciech Samek +2
The remarkable success of deep neural networks (DNNs) in various applications is accompanied by a significant increase in network parameters and arithmetic operations. Such increas…
FantastIC4: A Hardware-Software Co-Design Approach for Efficiently Running 4bit-Compact Multilayer Perceptrons
Simon Wiedemann, Suhas Shivapakash, Pablo Wiedemann +4
With the growing demand for deploying deep learning models to the "edge", it is paramount to develop techniques that allow to execute state-of-the-art models within very tight and…
Learning Sparse & Ternary Neural Networks with Entropy-Constrained Trained Ternarization (EC2T)
Arturo Marban, Daniel Becking, Simon Wiedemann +1
Deep neural networks (DNN) have shown remarkable success in a variety of machine learning applications. The capacity of these models (i.e., number of parameters), endows them with…