221 citations · 507 across the 42 of their papers we have counts for
28 papers · 1 filter
Beyond Low-rank Decomposition: A Shortcut Approach for Efficient On-Device Learning
Le-Trung Nguyen, Ael Quelennec, Van-Tam Nguyen +1
On-device learning has emerged as a promising direction for AI development, particularly because of its potential to reduce latency issues and mitigate privacy risks associated wit…
Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers
Zhu Liao, Nour Hezbri, Victor Quétu +2
Today, deep neural networks are widely used since they can handle a variety of complex tasks. Their generality makes them very powerful tools in modern technology. However, deep ne…
Activation Map Compression through Tensor Decomposition for Deep Learning
Le-Trung Nguyen, Aël Quélennec, Enzo Tartaglione +2
Internet of Things and Deep Learning are synergetically and exponentially growing industrial fields with a massive call for their unification into a common framework called Edge AI…
Memory-Optimized Once-For-All Network
Maxime Girard, Victor Quétu, Samuel Tardieu +2
Deploying Deep Neural Networks (DNNs) on different hardware platforms is challenging due to varying resource constraints. Besides handcrafted approaches aiming at making deep model…
LaCoOT: Layer Collapse through Optimal Transport
Victor Quétu, Zhu Liao, Nour Hezbri +2
Although deep neural networks are well-known for their outstanding performance in tackling complex tasks, their hunger for computational resources remains a significant hurdle, pos…
The Simpler The Better: An Entropy-Based Importance Metric To Reduce Neural Networks' Depth
Victor Quétu, Zhu Liao, Enzo Tartaglione
While deep neural networks are highly effective at solving complex tasks, large pre-trained models are commonly employed even to solve consistently simpler downstream tasks, which…
Layer Collapse Can be Induced by Unstructured Pruning
Zhu Liao, Victor Quétu, Van-Tam Nguyen +1
Unstructured pruning is a popular compression method for efficiently reducing model parameters. However, while it effectively decreases the number of parameters, it is commonly bel…
Debiasing surgeon: fantastic weights and how to find them
Rémi Nahon, Ivan Luiz De Moura Matos, Van-Tam Nguyen +1
Nowadays an ever-growing concerning phenomenon, the emergence of algorithmic biases that can lead to unfair models, emerges. Several debiasing approaches have been proposed in the…
SCoTTi: Save Computation at Training Time with an adaptive framework
Ziyu Lin, Enzo Tartaglione, Van-Tam Nguyen
On-device training is an emerging approach in machine learning where models are trained on edge devices, aiming to enhance privacy protection and real-time performance. However, ed…
Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraint
Aël Quélennec, Enzo Tartaglione, Pavlo Mozharovskyi +1
In the realm of efficient on-device learning under extreme memory and computation constraints, a significant gap in successful approaches persists. Although considerable effort has…
Can Unstructured Pruning Reduce the Depth in Deep Neural Networks?
Zhu Liao, Victor Quétu, Van-Tam Nguyen +1
Pruning is a widely used technique for reducing the size of deep neural networks while maintaining their performance. However, such a technique, despite being able to massively com…
Mining bias-target Alignment from Voronoi Cells
Rémi Nahon, Van-Tam Nguyen, Enzo Tartaglione
Despite significant research efforts, deep neural networks are still vulnerable to biases: this raises concerns about their fairness and limits their generalization. In this paper,…
Optimized preprocessing and Tiny ML for Attention State Classification
Yinghao Wang, Rémi Nahon, Enzo Tartaglione +2
In this paper, we present a new approach to mental state classification from EEG signals by combining signal processing techniques and machine learning (ML) algorithms. We evaluate…
DSD: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free?
Victor Quétu, Enzo Tartaglione
Neoteric works have shown that modern deep learning models can exhibit a sparse double descent phenomenon. Indeed, as the sparsity of the model increases, the test performance firs…
Can we avoid Double Descent in Deep Neural Networks?
Victor Quétu, Enzo Tartaglione
Finding the optimal size of deep learning models is very actual and of broad impact, especially in energy-saving schemes. Very recently, an unexpected phenomenon, the ``double desc…
Unbiased Supervised Contrastive Learning
Carlo Alberto Barbano, Benoit Dufumier, Enzo Tartaglione +2
Many datasets are biased, namely they contain easy-to-learn features that are highly correlated with the target class only in the dataset but not in the true underlying distributio…
Packed-Ensembles for Efficient Uncertainty Estimation
Olivier Laurent, Adrien Lafage, Enzo Tartaglione +4
Deep Ensembles (DE) are a prominent approach for achieving excellent performance on key metrics such as accuracy, calibration, uncertainty estimation, and out-of-distribution detec…
Information Removal at the bottleneck in Deep Neural Networks
Enzo Tartaglione
Deep learning models are nowadays broadly deployed to solve an incredibly large variety of tasks. Commonly, leveraging over the availability of "big data", deep neural networks are…
Disentangling private classes through regularization
Enzo Tartaglione, Francesca Gennari, Marco Grangetto
Deep learning models are nowadays broadly deployed to solve an incredibly large variety of tasks. However, little attention has been devoted to connected legal aspects. In 2016, th…
To update or not to update? Neurons at equilibrium in deep models
Andrea Bragagnolo, Enzo Tartaglione, Marco Grangetto
Recent advances in deep learning optimization showed that, with some a-posteriori information on fully-trained models, it is possible to match the same performance by simply traini…
Unsupervised Learning of Unbiased Visual Representations
Carlo Alberto Barbano, Enzo Tartaglione, Marco Grangetto
Deep neural networks often struggle to learn robust representations in the presence of dataset biases, leading to suboptimal generalization on unbiased datasets. This limitation ar…
The rise of the lottery heroes: why zero-shot pruning is hard
Enzo Tartaglione
Recent advances in deep learning optimization showed that just a subset of parameters are really necessary to successfully train a model. Potentially, such a discovery has broad im…
HEMP: High-order Entropy Minimization for neural network comPression
Enzo Tartaglione, Stéphane Lathuilière, Attilio Fiandrotti +2
We formulate the entropy of a quantized artificial neural network as a differentiable function that can be plugged as a regularization term into the cost function minimized by grad…
SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networks
Enzo Tartaglione, Andrea Bragagnolo, Francesco Odierna +2
Deep neural networks include millions of learnable parameters, making their deployment over resource-constrained devices problematic. SeReNe (Sensitivity-based Regularization of Ne…