TabRet: Pre-training Transformer-based Tabular Models for Unseen Columns

Soma Onishi, Kenta Oono, Kohei Hayashi

We present \emph{TabRet}, a pre-trainable Transformer-based model for tabular data. TabRet is designed to work on a downstream task that contains columns not seen in pre-training.…