Where Computation Lives Inside TabPFN: Causal Localisation of Attention Head Function

Atharva Gupta, Dhruv Kumar, Murari Mandal +1

We present the first causal mechanistic analysis of a tabular foundation model, investigating how TabPFN 2.5's feature wise attention heads distribute computation across layers. Us…