207 citations · 235 across the 22 of their papers we have counts for
13 papers · 1 filter
Inner Loop Inference for Pretrained Transformers: Unlocking Latent Capabilities Without Training
Jonathan Lys, Vincent Gripon, Bastien Pasdeloup +4
Deep Learning architectures, and in particular Transformers, are conventionally viewed as a composition of layers. These layers are actually often obtained as the sum of two contri…
GaLLoP: Gradient-based Sparse Learning on Low-Magnitude Parameters
Anand Choudhary, Yasser Sulaıman, Lukas Mauch +3
Sparse fine-tuning techniques adapt LLMs to downstream tasks by only tuning a sparse subset of model parameters. However, the effectiveness of sparse adaptation depends on optimall…
A Statistical Model for Predicting Generalization in Few-Shot Classification
Yassir Bendou, Vincent Gripon, Bastien Pasdeloup +5
The estimation of the generalization error of classifiers often relies on a validation set. Such a set is hardly available in few-shot learning scenarios, a highly disregarded shor…
Quantization and Deployment of Deep Neural Networks on Microcontrollers
Pierre-Emmanuel Novac, Ghouthi Boukli Hacene, Alain Pegatoquet +2
Embedding Artificial Intelligence onto low-power devices is a challenging task that has been partly overcome with recent advances in machine learning and hardware design. Presently…
Deeplite Neutrino: An End-to-End Framework for Constrained Deep Learning Model Optimization
Anush Sankaran, Olivier Mastropietro, Ehsan Saboori +4
Designing deep learning-based solutions is becoming a race for training deeper models with a greater number of layers. While a large-size deeper model could provide competitive acc…
DecisiveNets: Training Deep Associative Memories to Solve Complex Machine Learning Problems
Vincent Gripon, Carlos Lassance, Ghouthi Boukli Hacene
Learning deep representations to solve complex machine learning tasks has become the prominent trend in the past few years. Indeed, Deep Neural Networks are now the golden standard…
ThriftyNets : Convolutional Neural Networks with Tiny Parameter Budget
Guillaume Coiffier, Ghouthi Boukli Hacene, Vincent Gripon
Typical deep convolutional architectures present an increasing number of feature maps as we go deeper in the network, whereas spatial resolution of inputs is decreased through down…
BitPruning: Learning Bitlengths for Aggressive and Accurate Quantization
Miloš Nikolić, Ghouthi Boukli Hacene, Ciaran Bannon +5
Neural networks have demonstrably achieved state-of-the art accuracy using low-bitlength integer quantization, yielding both execution time and energy benefits on existing hardware…
Training Modern Deep Neural Networks for Memory-Fault Robustness
Ghouthi Boukli Hacene, François Leduc-Primeau, Amal Ben Soussia +2
Because deep neural networks (DNNs) rely on a large number of parameters and computations, their implementation in energy-constrained systems is challenging. In this paper, we inve…
Deep geometric knowledge distillation with graphs
Carlos Lassance, Myriam Bontonou, Ghouthi Boukli Hacene +3
In most cases deep learning architectures are trained disregarding the amount of operations and energy consumption. However, some applications, like embedded systems, can be resour…
Introducing Graph Smoothness Loss for Training Deep Learning Architectures
Myriam Bontonou, Carlos Lassance, Ghouthi Boukli Hacene +3
We introduce a novel loss function for training deep learning architectures to perform classification. It consists in minimizing the smoothness of label signals on similarity graph…
Quantized Guided Pruning for Efficient Hardware Implementations of Convolutional Neural Networks
Ghouthi Boukli Hacene, Vincent Gripon, Matthieu Arzel +2
Convolutional Neural Networks (CNNs) are state-of-the-art in numerous computer vision tasks such as object classification and detection. However, the large amount of parameters the…
Improving Accuracy of Nonparametric Transfer Learning via Vector Segmentation
Vincent Gripon, Ghouthi B. Hacene, Matthias Löwe +1
Transfer learning using deep neural networks as feature extractors has become increasingly popular over the past few years. It allows to obtain state-of-the-art accuracy on dataset…