11 papers
A Unifying Framework for Concept-Based Representational Similarity
Grégoire Dhimoïla, Victor Boutin, Agustin Martin Picard +2
Learned representations across models and modalities often exhibit striking structural similarities, suggesting shared underlying concept decompositions. However, concept alignment…
Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spot
Jorge Chang Ortega, Bastien Le Lan, Thomas Serre +1
A central question in computational vision is whether human-like visual representations are better explained by discriminative or generative learning. Existing comparisons, however…
Object-Level Explanations for Image Geolocation Models: a GeoGuessr use-case
Emilie Durrieu, Christophe Hurter, Philippe Muller +1
When humans play geolocation games such as GeoGuessr, they rely on concrete visual cues, such as road markings, vegetation, or architectural details, to infer where an image was ca…
Cross-Modal Redundancy and the Geometry of Vision-Language Embeddings
Grégoire Dhimoïla, Thomas Fel, Victor Boutin +1
Vision-language models (VLMs) align images and text with remarkable success, yet the geometry of their shared embedding space remains poorly understood. To probe this geometry, we…
Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Models
Louis Béthune, David Vigouroux, Yilun Du +3
What is the shortest path between two data points lying in a high-dimensional space? While the answer is trivial in Euclidean geometry, it becomes significantly more complex when t…
Deep Sturm--Liouville: From Sample-Based to 1D Regularization with Learnable Orthogonal Basis Functions
David Vigouroux, Joseba Dalmau, Louis Béthune +1
Although Artificial Neural Networks (ANNs) have achieved remarkable success across various tasks, they still suffer from limited generalization. We hypothesize that this limitation…
Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models
Thomas Fel, Ekdeep Singh Lubana, Jacob S. Prince +7
Sparse Autoencoders (SAEs) have emerged as a powerful framework for machine learning interpretability, enabling the unsupervised decomposition of model representations into a dicti…
ConSim: Measuring Concept-Based Explanations' Effectiveness with Automated Simulatability
Antonin Poché, Alon Jacovi, Agustin Martin Picard +2
Concept-based explanations work by mapping complex model computations to human-understandable concepts. Evaluating such explanations is very difficult, as it includes not only the…
Choosing the right basis for interpretability: Psychophysical comparison between neuron-based and dictionary-based representations
Julien Colin, Lore Goetschalckx, Thomas Fel +3
Interpretability research often adopts a neuron-centric lens, treating individual neurons as the fundamental units of explanation. However, neuron-level explanations can be undermi…
Latent Representation Matters: Human-like Sketches in One-shot Drawing Tasks
Victor Boutin, Rishav Mukherji, Aditya Agrawal +4
Humans can effortlessly draw new categories from a single exemplar, a feat that has long posed a challenge for generative models. However, this gap has started to close with recent…
Saliency strikes back: How filtering out high frequencies improves white-box explanations
Sabine Muzellec, Thomas Fel, Victor Boutin +3
Attribution methods correspond to a class of explainability methods (XAI) that aim to assess how individual inputs contribute to a model's decision-making process. We have identifi…