Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders

Chungpa Lee, Jihoon Kwon, Kyle Min +1

Vision-language models map images and text into a joint embedding space. However, these embeddings often entangle multiple semantic features, which limits their interpretability an…