4 papers · 1 filter
Where Computation Lives Inside TabPFN: Causal Localisation of Attention Head Function
Atharva Gupta, Dhruv Kumar, Murari Mandal +1
We present the first causal mechanistic analysis of a tabular foundation model, investigating how TabPFN 2.5's feature wise attention heads distribute computation across layers. Us…
Mix, Don't Pick: Why Synthetic Corpus Composition Matters for Time Series Foundation Model Pretraining
Aaryan Nagpal, Debdeep Sanyal, Murari Mandal +2
Choosing the wrong synthetic generator for time-series foundation model pretraining is costly: under identical training budgets, the best and worst generators produce up to a $2\ti…
REGEN: Reference-Guided Synthetic Multivariate Time Series Generation for Forecasting
Moulik Gupta, Dhruv Kumar, Murari Mandal +1
Training robust multivariate time series forecasting models requires large, diverse corpora, yet many real-world domains provide only a handful of observed sequences. Existing gene…
time2time: Causal Intervention in Hidden States to Simulate Rare Events in Time Series Foundation Models
Debdeep Sanyal, Aaryan Nagpal, Dhruv Kumar +2
While transformer-based foundation models excel at forecasting routine patterns, two questions remain: do they internalize semantic concepts such as market regimes, or merely fit c…