36 citations · 57 across the 22 of their papers we have counts for
28 papers
Learning Functional Subspaces for Neural Network Compression
Massimo Bini, Anders Christensen, Stephan Alaniz +5
Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keeping the matrices dense, and thus…
Look But Don't Touch with Sparse Autoencoders for Unlearning in Diffusion Models
Enrico Cassano, Riccardo Renzulli, Rayyan Ahmed +2
Sparse autoencoders (SAEs) have recently been proposed as interpretable tools for concept-level manipulation, under the assumption that isolated features can serve as controllable…
MIRAGE: Benchmarking and Aligning Multi-Instance Image Editing
Ziqian Liu, Stephan Alaniz
Instruction-guided image editing has seen remarkable progress with models like FLUX.2 and Qwen-Image-Edit, yet they still struggle with complex scenarios with multiple similar inst…
Explaining CLIP Zero-shot Predictions Through Concepts
Onat Ozdemir, Anders Christensen, Stephan Alaniz +2
Large-scale vision-language models such as CLIP have achieved remarkable success in zero-shot image recognition, yet their predictions remain largely opaque to human understanding.…
FINER: MLLMs Hallucinate under Fine-grained Negative Queries
Rui Xiao, Sanghwan Kim, Yongqin Xian +2
Multimodal large language models (MLLMs) struggle with hallucinations, particularly with fine-grained queries, a challenge underrepresented by existing benchmarks that focus on coa…
Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models
Leander Girrbach, Stephan Alaniz, Genevieve Smith +2
Vision-language models trained on large-scale multimodal datasets show strong demographic biases, but the role of training data in producing these biases remains unclear. A major b…
Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs
Sanghwan Kim, Rui Xiao, Stephan Alaniz +2
Multimodal Large Language Models (MLLMs) often struggle with fine-grained perception, such as identifying small objects in high-resolution images or detecting key moments in long v…
SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutions
Jessica Bader, Leander Girrbach, Stephan Alaniz +1
Concept Bottleneck Models (CBMs) and other concept-based interpretable models show great promise for making AI applications more transparent, which is essential in fields like medi…
LoFT: LoRA-fused Training Dataset Generation with Few-shot Guidance
Jae Myung Kim, Stephan Alaniz, Cordelia Schmid +1
Despite recent advances in text-to-image generation, using synthetically generated data seldom brings a significant boost in performance for supervised learning. Oftentimes, synthe…
Feasibility with Language Models for Open-World Compositional Zero-Shot Learning
Jae Myung Kim, Stephan Alaniz, Cordelia Schmid +1
Humans can easily tell if an attribute (also called state) is realistic, i.e., feasible, for an object, e.g. fire can be hot, but it cannot be wet. In Open-World Compositional Zero…
Concept-Guided Interpretability via Neural Chunking
Shuchen Wu, Stephan Alaniz, Shyamgopal Karthik +3
Neural networks are often described as black boxes, reflecting the significant challenge of understanding their internal workings and interactions. We propose a different perspecti…
A Large Scale Analysis of Gender Biases in Text-to-Image Generative Models
Leander Girrbach, Stephan Alaniz, Genevieve Smith +1
With the increasing use of image generation technology, understanding its social biases, including gender bias, is essential. This paper presents a large-scale study on gender bias…
Discovering Chunks in Neural Embeddings for Interpretability
Shuchen Wu, Stephan Alaniz, Eric Schulz +1
Understanding neural networks is challenging due to their high-dimensional, interacting components. Inspired by human cognition, which processes complex sensory data by chunking it…
FLAIR: VLM with Fine-grained Language-informed Image Representations
Rui Xiao, Sanghwan Kim, Mariana-Iuliana Georgescu +2
CLIP has shown impressive results in aligning images and texts at scale. However, its ability to capture detailed visual features remains limited because CLIP matches images and te…
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training
Sanghwan Kim, Rui Xiao, Mariana-Iuliana Georgescu +2
Vision-Language Models (VLMs) trained with contrastive loss have achieved significant advancements in various vision and language tasks. However, the global nature of the contrasti…
Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs)
Leander Girrbach, Stephan Alaniz, Yiran Huang +2
Pre-trained large language models (LLMs) have been reliably integrated with visual input for multimodal tasks. The widespread adoption of instruction-tuned image-to-text vision-lan…
DataDream: Few-shot Guided Dataset Generation
Jae Myung Kim, Jessica Bader, Stephan Alaniz +2
While text-to-image diffusion models have been shown to achieve state-of-the-art results in image synthesis, they have yet to prove their effectiveness in downstream applications.…
How should the advent of large language models affect the practice of science?
Marcel Binz, Stephan Alaniz, Adina Roskies +15
Large language models (LLMs) are being increasingly incorporated into scientific workflows. However, we have yet to fully grasp the implications of this integration. How should the…
PDiscoNet: Semantically consistent part discovery for fine-grained recognition
Robert van der Klis, Stephan Alaniz, Massimiliano Mancini +4
Fine-grained classification often requires recognizing specific object parts, such as beak shape and wing patterns for birds. Encouraging a fine-grained classification model to fir…
Iterative Superquadric Recomposition of 3D Objects from Multiple Views
Stephan Alaniz, Massimiliano Mancini, Zeynep Akata
Humans are good at recomposing novel objects, i.e. they can identify commonalities between unknown objects from general structure to finer detail, an ability difficult to replicate…
DeViL: Decoding Vision features into Language
Meghal Dani, Isabel Rio-Torto, Stephan Alaniz +1
Post-hoc explanation methods have often been criticised for abstracting away the decision-making process of deep neural networks. In this work, we would like to provide natural lan…
In-Context Impersonation Reveals Large Language Models' Strengths and Biases
Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto +2
In everyday conversations, humans can take on different roles and adapt their vocabulary to their chosen roles. We explore whether LLMs can take on, that is impersonate, different…
Semantic Image Synthesis with Semantically Coupled VQ-Model
Stephan Alaniz, Thomas Hummel, Zeynep Akata
Semantic image synthesis enables control over unconditional image generation by allowing guidance on what is being generated. We conditionally synthesize the latent space from a ve…
Abstracting Sketches through Simple Primitives
Stephan Alaniz, Massimiliano Mancini, Anjan Dutta +2
Humans show high-level of abstraction capabilities in games that require quickly communicating object information. They decompose the message content into multiple parts and commun…