6 citations · 11 across the 21 of their papers we have counts for
9 papers · 1 filter
Interpreting Physics in Video World Models
Sonia Joseph, Quentin Garrido, Randall Balestriero +5
A long-standing question in physical reasoning is whether video-based models need to rely on factorized representations of physical variables in order to make physically accurate p…
Bi-Orthogonal Factor Decomposition for Vision Transformers
Fenil R. Doshi, Thomas Fel, Talia Konkle +1
Self-attention is the central computational primitive of Vision Transformers, yet we lack a principled understanding of what information attention mechanisms exchange between token…
Block-Recurrent Dynamics in Vision Transformers
Mozes Jacobs, Thomas Fel, Richard Hakim +3
As Vision Transformers (ViTs) become standard vision backbones, a mechanistic account of their computational phenomenology is essential. Despite architectural cues that hint at dyn…
Into the Rabbit Hull: From Task-Relevant Concepts in DINO to Minkowski Geometry
Thomas Fel, Binxu Wang, Michael A. Lepori +8
DINOv2 is routinely deployed to recognize objects, scenes, and actions; yet the nature of what it perceives remains unknown. As a working baseline, we adopt the Linear Representati…
Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Models
Fenil R. Doshi, Thomas Fel, Talia Konkle +1
Humans are able to recognize objects based on both local texture cues and the configuration of object parts, yet contemporary vision models primarily harvest local texture cues, yi…
Interpreting the linear structure of vision-language model embedding spaces
Isabel Papadimitriou, Huangyuan Su, Thomas Fel +2
Vision-language models encode images and text in a joint space, minimizing the distance between corresponding image and text pairs. How are language and images organized in this jo…
Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models
Thomas Fel, Ekdeep Singh Lubana, Jacob S. Prince +7
Sparse Autoencoders (SAEs) have emerged as a powerful framework for machine learning interpretability, enabling the unsupervised decomposition of model representations into a dicti…
Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment
Harrish Thasarathan, Julian Forsyth, Thomas Fel +2
We present Universal Sparse Autoencoders (USAEs), a framework for uncovering and aligning interpretable concepts spanning multiple pretrained deep neural networks. Unlike existing…
Feature Accentuation: Revealing 'What' Features Respond to in Natural Images
Chris Hamblin, Thomas Fel, Srijani Saha +2
Efforts to decode neural network vision models necessitate a comprehensive grasp of both the spatial and semantic facets governing feature responses within images. Most research ha…