200 citations · 433 across the 16 of their papers we have counts for
8 papers · 1 filter
A White Paper on Neural Network Quantization
Markus Nagel, Marios Fournarakis, Rana Ali Amjad +3
While neural networks have advanced the frontiers in many applications, they often come at a high computational cost. Reducing the power and latency of neural network inference is…
Bayesian Bits: Unifying Quantization and Pruning
Mart van Baalen, Christos Louizos, Markus Nagel +4
We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization. Bayesian Bits employs a novel decomposition o…
Up or Down? Adaptive Rounding for Post-Training Quantization
Markus Nagel, Rana Ali Amjad, Mart van Baalen +2
When quantizing neural networks, assigning each floating-point weight to its nearest fixed-point value is the predominant approach. We find that, perhaps surprisingly, this is not…
Class-Conditional Compression and Disentanglement: Bridging the Gap between Neural Networks and Naive Bayes Classifiers
Rana Ali Amjad, Bernhard C. Geiger
In this draft, which reports on work in progress, we 1) adapt the information bottleneck functional by replacing the compression term by class-conditional compression, 2) relax thi…
Understanding Neural Networks and Individual Neuron Importance via Information-Ordered Cumulative Ablation
Rana Ali Amjad, Kairen Liu, Bernhard C. Geiger
In this work, we investigate the use of three information-theoretic quantities -- entropy, mutual information with the class variable, and a class selectivity measure based on Kull…
Extended Affinity Propagation: Global Discovery and Local Insights
Rayyan Ahmad Khan, Rana Ali Amjad, Martin Kleinsteuber
We propose a new clustering algorithm, Extended Affinity Propagation, based on pairwise similarities. Extended Affinity Propagation is developed by modifying Affinity Propagation s…
Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle
Rana Ali Amjad, Bernhard C. Geiger
In this theory paper, we investigate training deep neural networks (DNNs) for classification via minimizing the information bottleneck (IB) functional. We show that the resulting o…
Co-Clustering via Information-Theoretic Markov Aggregation
Clemens Bloechl, Rana Ali Amjad, Bernhard C. Geiger
We present an information-theoretic cost function for co-clustering, i.e., for simultaneous clustering of two sets based on similarities between their elements. By constructing a s…