2 citations · 3 across the 35 of their papers we have counts for
17 papers · 1 filter
Exploiting Biases in Attention for Transformers
Charles Herr, Gabriele Spadaro, Enzo Tartaglione
Attention is known to be sparse in vision transformers. In particular, we analyze the unequal relevance of attention-matrix columns to the final computation and show that this phen…
Validity-Preserving Hierarchical RL for Joint Routing and Switch Placement in EDA
Dorian Gailhard, Ugo Lecerf, Enzo Tartaglione +2
Routing and switch placement are fundamental combinatorial optimization problems in chip design, requiring the joint optimization of routing topology and physical placement under s…
From Deep to Shallow: Unconstrained and Efficient Layer Merging Strategy
Petro Shulzhenko, Gabriele Spadaro, Enzo Tartaglione
Although Deep Neural Networks have become foundational in many areas of Machine Learning, high computational demands limit their application in resource-constrained environments. T…
CutClean: Neural Network Pruning for Privacy-Preserving Inference
Leonardo Magliolo, Vito Paolo Pastore, Giuseppe Valenzise +1
Neural networks are increasingly deployed in high-stakes applications with growing privacy leakage concerns. We show that this privacy leakage can occur even in the absence of repr…
Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Models
Ivan Luiz De Moura Matos, Abdel Djalil Sad Saoud, Ekaterina Iakovleva +2
The issue of algorithmic biases in deep learning has led to the development of various debiasing techniques, many of which perform complex training procedures or dataset manipulati…
How I Met Your Bias: Investigating Bias Amplification in Diffusion Models
Nathan Roos, Ekaterina Iakovleva, Ani Gjergji +2
Diffusion-based generative models demonstrate state-of-the-art performance across various image synthesis tasks, yet their tendency to replicate and amplify dataset biases remains…
Memory Constrained Dynamic Subnetwork Update for Transfer Learning
Aël Quélennec, Pavlo Mozharovskyi, Van-Tam Nguyen +1
On-device neural network training faces critical memory constraints that limit the adaptation of pre-trained models to downstream tasks. We present MeDyate, a theoretically-grounde…
Study of Training Dynamics for Memory-Constrained Fine-Tuning
Aël Quélennec, Nour Hezbri, Pavlo Mozharovskyi +2
Memory-efficient training of deep neural networks has become increasingly important as models grow larger while deployment environments impose strict resource constraints. We propo…
Efficient Resource-Constrained Training of Transformers via Subspace Optimization
Le-Trung Nguyen, Enzo Tartaglione, Van-Tam Nguyen
As AI increasingly shapes daily life, energy consumption and data privacy have become pressing concerns. On-device learning trains models directly on edge devices, cutting energy c…
Neural Velocity for hyperparameter tuning
Gianluca Dalmasso, Andrea Bragagnolo, Enzo Tartaglione +2
Hyperparameter tuning, such as learning rate decay and defining a stopping criterion, often relies on monitoring the validation loss. This paper presents NeVe, a dynamic training a…
Feature-Aware (Hyper)graph Generation via Next-Scale Prediction
Dorian Gailhard, Enzo Tartaglione, Lirida Naviner +1
Graph generative models perform well on small-scale structured data but struggle to scale to large, complex structures. Hierarchical approaches improve scalability but often ignore…
Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing
Massimiliano Ciranni, Vito Paolo Pastore, Roberto Di Via +3
Deep learning model effectiveness in classification tasks is often challenged by the quality and quantity of training data whenever they are affected by strong spurious correlation…
WaterMAS: Sharpness-Aware Maximization for Neural Network Watermarking
Carl De Sousa Trias, Mihai Mitrea, Attilio Fiandrotti +3
Nowadays, deep neural networks are used for solving complex tasks in several critical applications and protecting both their integrity and intellectual property rights (IPR) has be…
HYGENE: A Diffusion-based Hypergraph Generation Method
Dorian Gailhard, Enzo Tartaglione, Lirida Naviner +1
Hypergraphs are powerful mathematical structures that can model complex, high-order relationships in various domains, including social networks, bioinformatics, and recommender sys…
Say My Name: a Model's Bias Discovery Framework
Massimiliano Ciranni, Luca Molinaro, Carlo Alberto Barbano +4
In the last few years, due to the broad applicability of deep learning to downstream tasks and end-to-end training capabilities, increasingly more concerns about potential biases t…
Find the Lady: Permutation and Re-Synchronization of Deep Neural Networks
Carl De Sousa Trias, Mihai Petru Mitrea, Attilio Fiandrotti +3
Deep neural networks are characterized by multiple symmetrical, equi-loss solutions that are redundant. Thus, the order of neurons in a layer and feature maps can be given arbitrar…
Weighted Ensemble Models Are Strong Continual Learners
Imad Eddine Marouf, Subhankar Roy, Enzo Tartaglione +1
In this work, we study the problem of continual learning (CL) where the goal is to learn a model on a sequence of tasks, such that the data from the previous tasks becomes unavaila…