4 papers
Characterizing Learning in Deep Neural Networks using Tractable Algorithmic Complexity Analysis
Pedram Bakhtiarifard, Sophia N. Wilson, Mahmoud Afifi +2
Training large-scale deep neural networks (DNNs) is resource-intensive, making model compression a practical necessity. The widely accepted ''learning as compression'' hypothesis p…
Algorithmic Simplification of Neural Networks with Mosaic-of-Motifs
Pedram Bakhtiarifard, Tong Chen, Jonathan Wenshøj +2
Large-scale deep learning models are well-suited for compression. Across a variety of tasks, methods like pruning, quantization, and knowledge distillation have been used to achiev…
CoDeQ: End-to-End Joint Model Compression with Dead-Zone Quantizer for High-Sparsity and Low-Precision Networks
Jonathan Wenshøj, Tong Chen, Bob Pepin +1
While joint pruning--quantization is theoretically superior to sequential application, current joint methods rely on auxiliary procedures outside the training loop for finding comp…
Oscillations Make Neural Networks Robust to Quantization
Jonathan Wenshøj, Bob Pepin, Raghavendra Selvan
We challenge the prevailing view that weight oscillations observed during Quantization Aware Training (QAT) are merely undesirable side-effects and argue instead that they are an e…