2 citations · 2 across the 3 of their papers we have counts for
3 papers
Sparser, Better, Deeper, Stronger: Improving Sparse Training with Exact Orthogonal Initialization
Aleksandra Irena Nowak, Łukasz Gniecki, Filip Szatkowski +1
Static sparse training aims to train sparse models from scratch, achieving remarkable results in recent years. A key design choice is given by the sparse initialization, which dete…
Adapt Your Teacher: Improving Knowledge Distillation for Exemplar-free Continual Learning
Filip Szatkowski, Mateusz Pyla, Marcin Przewięźlikowski +3
In this work, we investigate exemplar-free class incremental learning (CIL) with knowledge distillation (KD) as a regularization strategy, aiming to prevent forgetting. KD-based me…
Progressive Latent Replay for efficient Generative Rehearsal
Stanisław Pawlak, Filip Szatkowski, Michał Bortkiewicz +2
We introduce a new method for internal replay that modulates the frequency of rehearsal based on the depth of the network. While replay strategies mitigate the effects of catastrop…