Do Tabular Foundation Models Know Physics? Contamination, Units, and the Deterministic Limit

Abstract

Tabular foundation models (TFMs) learn to fill in tables the way language models fill in text --- and tables are arguably the format in which most physical measurement arrives. Did they learn any physics in the process? They are Bayesian by construction, so the question is what their prior contains. We probe it directly, evaluating four of them ($\text{TabPFN-3}$, $\text{TabICLv2}$, $\text{TabDPT}$ and $\text{Real-TabPFN-2.5}$) against six baselines on datasets sampled from 316 physical equations, in and out of domain. TFMs dominate, out of the box and after tuning. But they struggle to represent noiseless deterministic mechanisms in their uncertainty, and show no sign of exploiting physical units: they interpolate physics without yet being able to act as physical models.

arXiv comment5 pages (4 figures, 1 table), plus appendix. Accepted at the Representations for the Physical Sciences Workshop @ NeurIPS 2026