267 citations · 399 across the 9 of their papers we have counts for
9 papers · 1 filter
Learning Sparse & Ternary Neural Networks with Entropy-Constrained Trained Ternarization (EC2T)
Arturo Marban, Daniel Becking, Simon Wiedemann +1
Deep neural networks (DNN) have shown remarkable success in a variety of machine learning applications. The capacity of these models (i.e., number of parameters), endows them with…
Dithered backprop: A sparse and quantized backpropagation algorithm for more efficient deep neural network training
Simon Wiedemann, Temesgen Mehari, Kevin Kepp +1
Deep Neural Networks are successful but highly computationally expensive learning systems. One of the main sources of time and energy drains is the well known backpropagation (back…
Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning
Seul-Ki Yeom, Philipp Seegerer, Sebastian Lapuschkin +4
The success of convolutional neural networks (CNNs) in various applications is accompanied by a significant increase in computation and parameter storage costs. Recent efforts to r…
DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks
Simon Wiedemann, Heiner Kirchoffer, Stefan Matlage +10
The field of video compression has developed some of the most sophisticated and efficient compression algorithms known in the literature, enabling very high compressibility for lit…
DeepCABAC: Context-adaptive binary arithmetic coding for deep neural network compression
Simon Wiedemann, Heiner Kirchhoffer, Stefan Matlage +9
We present DeepCABAC, a novel context-adaptive binary arithmetic coder for compressing deep neural networks. It quantizes each weight parameter by minimizing a weighted rate-distor…
Robust and Communication-Efficient Federated Learning from Non-IID Data
Felix Sattler, Simon Wiedemann, Klaus-Robert Müller +1
Federated Learning allows multiple parties to jointly train a deep learning model on their combined data, without any of the participants having to reveal their local data to a cen…
Entropy-Constrained Training of Deep Neural Networks
Simon Wiedemann, Arturo Marban, Klaus-Robert Müller +1
We propose a general framework for neural network compression that is motivated by the Minimum Description Length (MDL) principle. For that we first derive an expression for the en…
Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication
Felix Sattler, Simon Wiedemann, Klaus-Robert Müller +1
Currently, progressively larger deep neural networks are trained on ever growing data corpora. As this trend is only going to increase in the future, distributed training schemes a…
Compact and Computationally Efficient Representation of Deep Neural Networks
Simon Wiedemann, Klaus-Robert Müller, Wojciech Samek
At the core of any inference procedure in deep neural networks are dot product operations, which are the component that require the highest computational resources. A common approa…