18 papers
TimEE: End-to-end Time Series Classification via In-Context Learning
Jaris Küken, Shi Bin Hoo, Martin Mráz +2
Time series classification (TSC) is dominated by a two-stage paradigm: train a feature encoder -- either from scratch on the target dataset or via pretraining on large corpora -- a…
Towards Evaluating Data Priors for Tabular Foundation Models
Zeynep Türkmen, KürÅat Kaya, Alexander Pfefferle +1
Data-generating priors are a central component of tabular foundation models because they define the task distribution used during pretraining. However, priors are rarely evaluated…
Selective Rotary Position Embedding
Sajad Movahedi, Timur Carstensen, Arshia Afzal +3
Position information is essential for language modeling. In softmax transformers, Rotary Position Embeddings (\textit{RoPE}) encode positions through \textit{fixed-angle} rotations…
Towards Pretraining Text Encoders for TabPFN
Mustafa Tajjar, Alexander Pfefferle, Lennart Purucker +1
Tabular foundation models, such as TabPFN, achieve strong performance on tabular datasets with numerical and categorical data, but do not natively handle high-cardinality text feat…
TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models
Léo Grinsztajn, Klemens Flöge, Oscar Key +23
The first tabular foundation model, TabPFN, and its successor TabPFNv2 have impacted tabular AI substantially, with dozens of methods building on it and hundreds of applications ac…
TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting
Vladyslav Moroshan, Julien Siems, Arber Zela +2
Foundation models for zero-shot time series forecasting face challenges in efficient long-horizon prediction and reproducibility, with existing synthetic-only approaches underperfo…