5 citations · 11 across the 6 of their papers we have counts for
6 papers
Stiefel-AdamW: Geometry-Aware AdamW for Linear Factorization Blocks
Emanuele Zangrando, Marco Sutti, Francesco Tudisco
A pervasive structural pattern in modern deep learning is the linear factorization block: a submodule of the form in which two parameter matrices are multiplied directly,…
GeoLoRA: Geometric integration for parameter efficient fine-tuning
Steffen Schotthöfer, Emanuele Zangrando, Gianluca Ceruti +2
Low-Rank Adaptation (LoRA) has become a widely used method for parameter-efficient fine-tuning of large-scale, pre-trained neural networks. However, LoRA and its extensions face se…
Low-Rank Adversarial PGD Attack
Dayana Savostianova, Emanuele Zangrando, Francesco Tudisco
Adversarial attacks on deep neural network models have seen rapid development and are extensively used to study the stability of these networks. Among various adversarial strategie…
Robust low-rank training via approximate orthonormal constraints
Dayana Savostianova, Emanuele Zangrando, Gianluca Ceruti +1
With the growth of model and data sizes, a broad effort has been made to design pruning techniques that reduce the resource demand of deep learning pipelines, while retaining model…
Geometry-aware training of factorized layers in tensor Tucker format
Emanuele Zangrando, Steffen Schotthöfer, Gianluca Ceruti +2
Reducing parameter redundancies in neural network architectures is crucial for achieving feasible computational and memory requirements during training and inference phases. Given…
Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations
Steffen Schotthöfer, Emanuele Zangrando, Jonas Kusch +2
Neural networks have achieved tremendous success in a large variety of applications. However, their memory footprint and computational demand can render them impractical in applica…