6 papers
TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning
Yury Gorishniy, Akim Kotelnikov, Ivan Rubachev +1
In deep learning for tabular data, efficient ensembles of multilayer perceptrons (MLPs) have recently emerged as effective and practical architectures. Existing methods of this kin…
Talking Trees: Reasoning-Assisted Induction of Decision Trees for Tabular Data
George Yakushev, Alina Shutova, Ivan Rubachev +3
Tabular foundation models are becoming increasingly popular for low-resource tabular problems. These models make up for small training datasets by pretraining on large volumes of s…
Benchmarking Optimizers for MLPs in Tabular Deep Learning
Yury Gorishniy, Ivan Rubachev, Dmitrii Feoktistov +1
MLP is a heavily used backbone in modern deep learning (DL) architectures for supervised learning on tabular data, and AdamW is the go-to optimizer used to train tabular DL models.…
Unveiling the Role of Data Uncertainty in Tabular Deep Learning
Nikolay Kartashev, Ivan Rubachev, Artem Babenko
Recent advancements in tabular deep learning have demonstrated exceptional practical performance, yet the field often lacks a clear understanding of why these techniques actually s…
On Finetuning Tabular Foundation Models
Ivan Rubachev, Akim Kotelnikov, Nikolay Kartashev +1
Foundation models are an emerging research direction in tabular deep learning. Notably, TabPFNv2 recently claimed superior performance over traditional GBDT-based methods on small-…
TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling
Yury Gorishniy, Akim Kotelnikov, Artem Babenko
Deep learning architectures for supervised learning on tabular data range from simple multilayer perceptrons (MLP) to sophisticated Transformers and retrieval-augmented methods. Th…