7 citations · 16 across the 12 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024
Towards Optimal Adapter Placement for Efficient Transfer Learning
Aleksandra I. Nowak, Otniel-Bogdan Mercea, Anurag Arnab +3
Parameter-efficient transfer learning (PETL) aims to adapt pre-trained models to new downstream tasks while minimizing the number of fine-tuned parameters. Adapters, a popular appr…
cs.LG2024
Sparser, Better, Deeper, Stronger: Improving Sparse Training with Exact Orthogonal Initialization
Aleksandra Irena Nowak, Łukasz Gniecki, Filip Szatkowski +1
Static sparse training aims to train sparse models from scratch, achieving remarkable results in recent years. A key design choice is given by the sparse initialization, which dete…