25 citations · 43 across the 12 of their papers we have counts for
13 papers
HASTE: A Framework for Training-Free, Dynamic, and Steerable Compression of Pre-Trained Convolutional Neural Networks
Lukas Meiner, Jens Mehnert, Alexandru Paul Condurache
Deploying large convolutional neural networks (CNNs) on resource-constrained devices is challenging due to their high computational cost. While dynamic execution methods are promis…
Variance-Based Pruning for Accelerating and Compressing Trained Networks
Uranik Berisha, Jens Mehnert, Alexandru Paul Condurache
Increasingly expensive training of ever larger models such as Vision Transfomers motivate reusing the vast library of already trained state-of-the-art networks. However, their late…
Efficient Data Driven Mixture-of-Expert Extraction from Trained Networks
Uranik Berisha, Jens Mehnert, Alexandru Paul Condurache
Vision Transformers have emerged as the state-of-the-art models in various Computer Vision tasks, but their high computational and resource demands pose significant challenges. Whi…
PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNs
Lukas Meiner, Jens Mehnert, Alexandru Paul Condurache
Convolutional neural networks (CNNs) are crucial for computer vision tasks on resource-constrained devices. Quantization effectively compresses these models, reducing storage size…
Squeeze-and-Remember Block
Rinor Cakaj, Jens Mehnert, Bin Yang
Convolutional Neural Networks (CNNs) are important for many machine learning tasks. They are built with different types of layers: convolutional layers that detect features, dropou…
Spectral Wavelet Dropout: Regularization in the Wavelet Domain
Rinor Cakaj, Jens Mehnert, Bin Yang
Regularization techniques help prevent overfitting and therefore improve the ability of convolutional neural networks (CNNs) to generalize. One reason for overfitting is the comple…
CNN Mixture-of-Depths
Rinor Cakaj, Jens Mehnert, Bin Yang
We introduce Mixture-of-Depths (MoD) for Convolutional Neural Networks (CNNs), a novel approach that enhances the computational efficiency of CNNs by selectively processing channel…
Data-Free Dynamic Compression of CNNs for Tractable Efficiency
Lukas Meiner, Jens Mehnert, Alexandru Paul Condurache
To reduce the computational cost of convolutional neural networks (CNNs) on resource-constrained devices, structured pruning approaches have shown promise in lowering floating-poin…
Spectral Batch Normalization: Normalization in the Frequency Domain
Rinor Cakaj, Jens Mehnert, Bin Yang
Regularization is a set of techniques that are used to improve the generalization ability of deep neural networks. In this paper, we introduce spectral batch normalization (SBN), a…
Weight Compander: A Simple Weight Reparameterization for Regularization
Rinor Cakaj, Jens Mehnert, Bin Yang
Regularization is a set of techniques that are used to improve the generalization ability of deep neural networks. In this paper, we introduce weight compander (WC), a novel effect…
Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey
Paul Wimmer, Jens Mehnert, Alexandru Paul Condurache
State-of-the-art deep learning models have a parameter count that reaches into the billions. Training, storing and transferring such models is energy and time consuming, thus costl…
Interspace Pruning: Using Adaptive Filter Representations to Improve Training of Sparse CNNs
Paul Wimmer, Jens Mehnert, Alexandru Paul Condurache
Unstructured pruning is well suited to reduce the memory footprint of convolutional neural networks (CNNs), both at training and inference time. CNNs contain parameters arranged in…
FreezeNet: Full Performance by Reduced Storage Costs
Paul Wimmer, Jens Mehnert, Alexandru Condurache
Pruning generates sparse networks by setting parameters to zero. In this work we improve one-shot pruning methods, applied before training, without adding any additional storage co…