7 citations · 12 across the 5 of their papers we have counts for
10 papers · 1 filter
Towards Optimal Adapter Placement for Efficient Transfer Learning
Aleksandra I. Nowak, Otniel-Bogdan Mercea, Anurag Arnab +3
Parameter-efficient transfer learning (PETL) aims to adapt pre-trained models to new downstream tasks while minimizing the number of fine-tuned parameters. Adapters, a popular appr…
Sparser, Better, Deeper, Stronger: Improving Sparse Training with Exact Orthogonal Initialization
Aleksandra Irena Nowak, Łukasz Gniecki, Filip Szatkowski +1
Static sparse training aims to train sparse models from scratch, achieving remarkable results in recent years. A key design choice is given by the sparse initialization, which dete…
Fantastic Weights and How to Find Them: Where to Prune in Dynamic Sparse Training
Aleksandra I. Nowak, Bram Grooten, Decebal Constantin Mocanu +1
Dynamic Sparse Training (DST) is a rapidly evolving area of research that seeks to optimize the sparse initialization of a neural network by adapting its topology during training.…
Non-Gaussian Gaussian Processes for Few-Shot Regression
Marcin Sendera, Jacek Tabor, Aleksandra Nowak +5
Gaussian Processes (GPs) have been widely used in machine learning to model distributions over functions, with applications including multi-modal regression, time-series prediction…
On the relationship between disentanglement and multi-task learning
Łukasz Maziarka, Aleksandra Nowak, Maciej Wołczyk +1
One of the main arguments behind studying disentangled representations is the assumption that they can be easily reused in different tasks. At the same time finding a joint, adapta…
Neural networks adapting to datasets: learning network size and topology
Romuald A. Janik, Aleksandra Nowak
We introduce a flexible setup allowing for a neural network to learn both its size and topology during the course of a standard gradient-based training. The resulting network has t…