5 citations · 9 across the 14 of their papers we have counts for
14 papers
Shieldstral
Antonia Calvi, Avinash Sooriyarachchi, Giada Pistilli +273
We introduce Shieldstral, a 3B-parameter policy-adaptive multimodal safety classifier that matches or outperforms models nearly 7 its size on text safety benchmarks and set…
Beyond Classification: Dynamic Adapter Routing for Continual Multimodal Retrieval
Alicja Dobrzeniecka, Filip Szatkowski, Sebastian Cygert +2
While retrieval is a core function of vision-language models, continually updating these models for retrieval tasks remains critically underexplored. Existing work often approaches…
Universal Properties of Activation Sparsity in Modern Large Language Models
Filip Szatkowski, Patryk Będkowski, Alessio Devoto +5
Activation sparsity is an intriguing property of deep neural networks that has been extensively studied in ReLU-based models, due to its advantages for efficiency, robustness, and…
ExpertSim: Fast Particle Detector Simulation Using Mixture-of-Generative-Experts
Patryk Będkowski, Jan Dubiński, Filip Szatkowski +3
Simulating detector responses is a crucial part of understanding the inner workings of particle collisions in the Large Hadron Collider at CERN. Such simulations are currently perf…
Rethinking Calibration for Early-Exit Neural Networks
Piotr Kubaty, Filip Szatkowski, Grzegorz Choczyński +2
Early-exit neural networks (EENNs) accelerate inference by allowing intermediate classifiers to stop computation once predictions are confident enough. Most methods rely on confide…
Efficient LLM Moderation with Multi-Layer Latent Prototypes
Maciej Chrabąszcz, Filip Szatkowski, Bartosz Wójcik +3
Although modern LLMs are aligned with human values during post-training, robust moderation remains essential to prevent harmful outputs at deployment time. Existing approaches suff…
Exploring the Stability Gap in Continual Learning: The Role of the Classification Head
Wojciech Łapacz, Daniel Marczak, Filip Szatkowski +1
Continual learning (CL) has emerged as a critical area in machine learning, enabling neural networks to learn from evolving data distributions while mitigating catastrophic forgett…
Sparser, Better, Deeper, Stronger: Improving Sparse Training with Exact Orthogonal Initialization
Aleksandra Irena Nowak, Łukasz Gniecki, Filip Szatkowski +1
Static sparse training aims to train sparse models from scratch, achieving remarkable results in recent years. A key design choice is given by the sparse initialization, which dete…
Improving Continual Learning Performance and Efficiency with Auxiliary Classifiers
Filip Szatkowski, Yaoyue Zheng, Fei Yang +3
Continual learning is crucial for applying machine learning in challenging, dynamic, and often resource-constrained environments. However, catastrophic forgetting - overwriting pre…
Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversion
Filip Szatkowski, Bartosz Wójcik, Mikołaj Piórczyński +1
Transformer models can face practical limitations due to their high computational requirements. At the same time, such models exhibit significant activation sparsity, which can be…
Adapt Your Teacher: Improving Knowledge Distillation for Exemplar-free Continual Learning
Filip Szatkowski, Mateusz Pyla, Marcin Przewięźlikowski +3
In this work, we investigate exemplar-free class incremental learning (CIL) with knowledge distillation (KD) as a regularization strategy, aiming to prevent forgetting. KD-based me…
Hypernetworks build Implicit Neural Representations of Sounds
Filip Szatkowski, Karol J. Piczak, Przemysław Spurek +2
Implicit Neural Representations (INRs) are nowadays used to represent multimedia signals across various real-life applications, including image super-resolution, image compression,…
HyperSound: Generating Implicit Neural Representations of Audio Signals with Hypernetworks
Filip Szatkowski, Karol J. Piczak, Przemysław Spurek +2
Implicit neural representations (INRs) are a rapidly growing research field, which provides alternative ways to represent multimedia signals. Recent applications of INRs include im…
Progressive Latent Replay for efficient Generative Rehearsal
Stanisław Pawlak, Filip Szatkowski, Michał Bortkiewicz +2
We introduce a new method for internal replay that modulates the frequency of rehearsal based on the depth of the network. While replay strategies mitigate the effects of catastrop…