58 citations · 163 across the 41 of their papers we have counts for
48 papers
On the Limits of Representation-Based Test-Time Adaptation for Multimodal LLMs
Mikołaj Piórczyński, Wojciech Masarczyk, Damian Sójka +2
Vision-language models suffer significant accuracy drops under common corruptions. While test-time adaptation (TTA) successfully mitigates these shifts in CLIP-like contrastive mod…
Does Physics Live in the Activations? Localizing Physical Quantities in Video Diffusion Models
Jonas Kneifl, Jakub Skalski, Bartłomiej Twardowski +1
Video generation models produce strikingly realistic sequences and are increasingly proposed as world models, yet recent benchmarks reveal pronounced deficits in their physical rea…
Rethinking the Teacher-Student Framework for Test-Time Adaptation
Damian Sójka, Marc Masana, Bartłomiej Twardowski +1
Test-Time Adaptation (TTA) has recently emerged as a promising strategy that allows the adaptation of pre-trained models to changing data distributions at deployment time, without…
Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models
Sergi Masip, Alicja Dobrzeniecka, Jonathan Swinnen +4
Traditionally, continual learning has assumed access to labeled data, yet many real-world applications -- such as lifelong robotics -- require models to adapt continuously from unl…
Beyond Classification: Dynamic Adapter Routing for Continual Multimodal Retrieval
Alicja Dobrzeniecka, Filip Szatkowski, Sebastian Cygert +2
While retrieval is a core function of vision-language models, continually updating these models for retrieval tasks remains critically underexplored. Existing work often approaches…
Online Continual Learning with Dynamic Label Hierarchies
Xinrui Wang, Shao-Yuan Li, Bartłomiej Twardowski +2
Online Continual Learning (OCL) aims to learn from endless non\text{-}stationary data streams, yet most existing methods assume a flat label space and overlook the hierarchical org…
Project and Mix: Task-Semantic Prototypes for Few-Shot Image Classification
Dipam Goswami, Simone Magistri, Gido M. van de Ven +4
Vision-language models like CLIP are trained with the objective of aligning text and image pairs. Beyond text prompts alone, recent works show that exploiting few-shot image embedd…
IsoCLIP: Decomposing CLIP Projectors for Efficient Intra-modal Alignment
Simone Magistri, Dipam Goswami, Marco Mistretta +3
Vision-Language Models like CLIP are extensively used for inter-modal tasks which involve both visual and text modalities. However, when the individual modality encoders are applie…
Backdoor Vectors: a Task Arithmetic View on Backdoor Attacks and Defenses
Stanisław Pawlak, Jan Dubiński, Daniel Marczak +1
Model merging (MM) recently emerged as an effective method for combining large deep learning models. However, it poses significant security risks. Recent research shows that it is…
Accurate and Efficient Low-Rank Model Merging in Core Space
Aniello Panariello, Daniel Marczak, Simone Magistri +5
In this paper, we address the challenges associated with merging low-rank adaptations of large neural networks. With the rise of parameter-efficient adaptation techniques, such as…
Addressing The Devastating Effects Of Single-Task Data Poisoning In Exemplar-Free Continual Learning
Stanisław Pawlak, Bartłomiej Twardowski, Tomasz Trzciński +1
Our research addresses the overlooked security concerns related to data poisoning in continual learning (CL). Data poisoning - the intentional manipulation of training data to affe…
Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Models
Dipam Goswami, Liying Wang, Bartłomiej Twardowski +1
Text embedding models enable semantic search, powering several NLP applications like Retrieval Augmented Generation by efficient information retrieval (IR). However, text embedding…
No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces
Daniel Marczak, Simone Magistri, Sebastian Cygert +3
Model merging integrates the weights of multiple task-specific models into a single multi-task model. Despite recent interest in the problem, a significant performance gap between…
The Art of Deception: Color Visual Illusions and Diffusion Models
Alex Gomez-Villa, Kai Wang, Alejandro C. Parraga +4
Visual illusions in humans arise when interpreting out-of-distribution stimuli: if the observer is adapted to certain statistics, perception of outliers deviates from reality. Rece…
Covariances for Free: Exploiting Mean Distributions for Training-free Federated Learning
Dipam Goswami, Simone Magistri, Kai Wang +3
Using pre-trained models has been found to reduce the effect of data heterogeneity and speed up federated learning algorithms. Recent works have explored training-free methods usin…
Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning
Grzegorz Rypeść, Sebastian Cygert, Tomasz Trzciński +1
Exemplar-Free Class Incremental Learning (EFCIL) tackles the problem of training a model on a sequence of tasks without access to past data. Existing state-of-the-art methods repre…
MagMax: Leveraging Model Merging for Seamless Continual Learning
Daniel Marczak, Bartłomiej Twardowski, Tomasz Trzciński +1
This paper introduces a continual learning approach named MagMax, which utilizes model merging to enable large pre-trained models to continuously learn from new data without forget…
Realistic Evaluation of Test-Time Adaptation Algorithms: Unsupervised Hyperparameter Selection
Sebastian Cygert, Damian Sójka, Tomasz Trzciński +1
Test-Time Adaptation (TTA) has recently emerged as a promising strategy for tackling the problem of machine learning model robustness under distribution shifts by adapting the mode…
Exemplar-free Continual Representation Learning via Learnable Drift Compensation
Alex Gomez-Villa, Dipam Goswami, Kai Wang +3
Exemplar-free class-incremental learning using a backbone trained from scratch and starting from a small first task presents a significant challenge for continual representation le…
Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning
Dipam Goswami, Albin Soutif--Cormerais, Yuyang Liu +3
Continual learning methods are known to suffer from catastrophic forgetting, a phenomenon that is particularly hard to counter for methods that do not store exemplars of previous t…
Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformers
Dipam Goswami, Bartłomiej Twardowski, Joost van de Weijer
Few-shot class-incremental learning (FSCIL) aims to adapt the model to new classes from very few data (5 samples) without forgetting the previously learned classes. Recent works in…
Improving Continual Learning Performance and Efficiency with Auxiliary Classifiers
Filip Szatkowski, Yaoyue Zheng, Fei Yang +3
Continual learning is crucial for applying machine learning in challenging, dynamic, and often resource-constrained environments. However, catastrophic forgetting - overwriting pre…
GUIDE: Guidance-based Incremental Learning with Diffusion Models
Bartosz Cywiński, Kamil Deja, Tomasz Trzciński +2
We introduce GUIDE, a novel continual learning approach that directs diffusion models to rehearse samples at risk of being forgotten. Existing generative strategies combat catastro…
Divide and not forget: Ensemble of selectively trained experts in Continual Learning
Grzegorz Rypeść, Sebastian Cygert, Valeriya Khan +3
Class-incremental learning is becoming more popular as it helps models widen their applicability while not forgetting what they already know. A trend in this area is to use a mixtu…