81 citations · 429 across the 42 of their papers we have counts for
8 papers · 1 filter
Distilling Optimal Neural Networks: Rapid Search in Diverse Spaces
Bert Moons, Parham Noorzad, Andrii Skliar +4
Current state-of-the-art Neural Architecture Search (NAS) methods neither efficiently scale to multiple hardware platforms, nor handle diverse architectural search-spaces. To remed…
Differentiable Joint Pruning and Quantization for Hardware Efficiency
Ying Wang, Yadong Lu, Tijmen Blankevoort
We present a differentiable joint pruning and quantization (DJPQ) scheme. We frame neural network compression as a joint gradient-based optimization problem, trading off between mo…
Bayesian Bits: Unifying Quantization and Pruning
Mart van Baalen, Christos Louizos, Markus Nagel +4
We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization. Bayesian Bits employs a novel decomposition o…
LSQ+: Improving low-bit quantization through learnable offsets and better initialization
Yash Bhalgat, Jinwon Lee, Markus Nagel +2
Unlike ReLU, newer activation functions (like Swish, H-swish, Mish) that are frequently employed in popular efficient architectures can also result in negative activation values, w…
Up or Down? Adaptive Rounding for Post-Training Quantization
Markus Nagel, Rana Ali Amjad, Mart van Baalen +2
When quantizing neural networks, assigning each floating-point weight to its nearest fixed-point value is the predominant approach. We find that, perhaps surprisingly, this is not…
Conditional Channel Gated Networks for Task-Aware Continual Learning
Davide Abati, Jakub Tomczak, Tijmen Blankevoort +3
Convolutional Neural Networks experience catastrophic forgetting when optimized on a sequence of learning problems: as they meet the objective of the current training examples, the…