18 citations · 58 across the 32 of their papers we have counts for
39 papers
Project and Mix: Task-Semantic Prototypes for Few-Shot Image Classification
Dipam Goswami, Simone Magistri, Gido M. van de Ven +4
Vision-language models like CLIP are trained with the objective of aligning text and image pairs. Beyond text prompts alone, recent works show that exploiting few-shot image embedd…
NumColor: Precise Numeric Color Control in Text-to-Image Generation
Muhammad Atif Butt, Diego Hernandez, Alexandra Gomez-Villa +3
Text-to-image diffusion models excel at generating images from natural language descriptions, yet fail to interpret numerical colors such as hex codes (#FF5733) and RGB values (rgb…
IsoCLIP: Decomposing CLIP Projectors for Efficient Intra-modal Alignment
Simone Magistri, Dipam Goswami, Marco Mistretta +3
Vision-Language Models like CLIP are extensively used for inter-modal tasks which involve both visual and text modalities. However, when the individual modality encoders are applie…
Position: Modular Memory is the Key to Continual Learning Agents
Vaggelis Dorovatas, Malte Schwerin, Andrew D. Bagdanov +21
Foundation models have transformed machine learning through large-scale pretraining and increased test-time compute. Despite surpassing human performance in several domains, these…
Revisiting Weight Regularization for Low-Rank Continual Learning
Yaoyue Zheng, Yin Zhang, Joost van de Weijer +4
Continual Learning (CL) with large-scale pre-trained models (PTMs) has recently gained wide attention, shifting the focus from training from scratch to continually adapting PTMs. T…
LumiCtrl : Learning Illuminant Prompts for Lighting Control in Personalized Text-to-Image Models
Muhammad Atif Butt, Kai Wang, Javier Vazquez-Corral +1
Text-to-image (T2I) models have demonstrated remarkable progress in creative image generation, yet they still lack precise control over scene illuminants which is a crucial factor…
Adversarial Concept Distillation for One-Step Diffusion Personalization
Yixiong Yang, Tao Wu, Senmao Li +4
Recent progress in accelerating text-to-image diffusion models enables high-fidelity synthesis within a single denoising step. However, customizing the fast one-step models remains…
Accurate and Efficient Low-Rank Model Merging in Core Space
Aniello Panariello, Daniel Marczak, Simone Magistri +5
In this paper, we address the challenges associated with merging low-rank adaptations of large neural networks. With the rise of parameter-efficient adaptation techniques, such as…
Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting
Yuyang Liu, Qiuhe Hong, Linlan Huang +6
Vision-language models (VLMs), spanning predictive architectures to generative Multimodal Large Language Models (MLLMs), have revolutionized artificial intelligence through powerfu…
CLA: Latent Alignment for Online Continual Self-Supervised Learning
Giacomo Cignoni, Andrea Cossu, Alexandra Gomez-Villa +2
Self-supervised learning (SSL) is able to build latent representations that generalize well to unseen data. However, only a few SSL techniques exist for the online CL setting, wher…
Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Models
Dipam Goswami, Liying Wang, Bartłomiej Twardowski +1
Text embedding models enable semantic search, powering several NLP applications like Retrieval Augmented Generation by efficient information retrieval (IR). However, text embedding…
One-Way Ticket:Time-Independent Unified Encoder for Distilling Text-to-Image Diffusion Models
Senmao Li, Lei Wang, Kai Wang +7
Text-to-Image (T2I) diffusion models have made remarkable advancements in generative modeling; however, they face a trade-off between inference speed and image quality, posing chal…
InterLCM: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration
Senmao Li, Kai Wang, Joost van de Weijer +6
Diffusion priors have been used for blind face restoration (BFR) by fine-tuning diffusion models (DMs) on restoration datasets to recover low-quality images. However, the naive app…
Free-Lunch Color-Texture Disentanglement for Stylized Image Generation
Jiang Qin, Senmao Li, Alexandra Gomez-Villa +4
Recent advances in Text-to-Image (T2I) diffusion models have transformed image generation, enabling significant progress in stylized generation using only a few style reference ima…
EFC++: Elastic Feature Consolidation with Prototype Re-balancing for Cold Start Exemplar-free Incremental Learning
Simone Magistri, Tomaso Trinci, Albin Soutif-Cormerais +2
Exemplar-free Class Incremental Learning (EFCIL) aims to learn from a sequence of tasks without having access to previous task data. In this paper, we consider the challenging Cold…
Leveraging Semantic Attribute Binding for Free-Lunch Color Control in Diffusion Models
Héctor Laria, Alexandra Gomez-Villa, Jiang Qin +5
Recent advances in text-to-image (T2I) diffusion models have enabled remarkable control over various attributes, yet precise color specification remains a fundamental challenge. Ex…
Replay-free Online Continual Learning with Self-Supervised MultiPatches
Giacomo Cignoni, Andrea Cossu, Alex Gomez-Villa +2
Online Continual Learning (OCL) methods train a model on a non-stationary data stream where only a few examples are available at a time, often leveraging replay strategies. However…
One-Prompt-One-Story: Free-Lunch Consistent Text-to-Image Generation Using a Single Prompt
Tao Liu, Kai Wang, Senmao Li +6
Text-to-image generation models can create high-quality images from input prompts. However, they struggle to support the consistent generation of identity-preserving requirements f…
No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces
Daniel Marczak, Simone Magistri, Sebastian Cygert +3
Model merging integrates the weights of multiple task-specific models into a single multi-task model. Despite recent interest in the problem, a significant performance gap between…
Covariances for Free: Exploiting Mean Distributions for Training-free Federated Learning
Dipam Goswami, Simone Magistri, Kai Wang +3
Using pre-trained models has been found to reduce the effect of data heterogeneity and speed up federated learning algorithms. Recent works have explored training-free methods usin…
Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesis
Taihang Hu, Linxuan Li, Joost van de Weijer +6
Although text-to-image (T2I) models exhibit remarkable generation capabilities, they frequently fail to accurately bind semantically related objects or attributes in the input prom…
Multi-Class Textual-Inversion Secretly Yields a Semantic-Agnostic Classifier
Kai Wang, Fei Yang, Bogdan Raducanu +1
With the advent of large pre-trained vision-language models such as CLIP, prompt learning methods aim to enhance the transferability of the CLIP model. They learn the prompt given…
Assessing Open-world Forgetting in Generative Image Model Customization
Héctor Laria, Alex Gomez-Villa, Kai Wang +2
Recent advances in diffusion models have significantly enhanced image generation capabilities. However, customizing these models with new classes often leads to unintended conseque…
Exemplar-free Continual Representation Learning via Learnable Drift Compensation
Alex Gomez-Villa, Dipam Goswami, Kai Wang +3
Exemplar-free class-incremental learning using a backbone trained from scratch and starting from a small first task presents a significant challenge for continual representation le…