Simple Modifications to Improve Tabular Neural Networks
arXiv:2108.03214
Abstract
There is growing interest in neural network architectures for tabular data. Many general-purpose tabular deep learning models have been introduced recently, with performance sometimes rivaling gradient boosted decision trees (GBDTs). These recent models draw inspiration from various sources, including GBDTs, factorization machines, and neural networks from other application domains. Previous tabular neural networks are also drawn upon, but are possibly under-considered, especially models associated with specific tabular problems. This paper focuses on several such models, and proposes modifications for improving their performance. When modified, these models are shown to be competitive with leading general-purpose tabular models, including GBDTs.
References in corpus (10)
- mixup: Beyond Empirical Risk Minimization
- Improved Regularization of Convolutional Neural Networks with Cutout
- Train longer, generalize better: closing the generalization gap in large batch training of neural networks
- TabTransformer: Tabular Data Modeling Using Contextual Embeddings
- AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data
- Revisiting Deep Learning Models for Tabular Data
- Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data
- Product-based Neural Networks for User Response Prediction
- DNF-Net: A Neural Architecture for Tabular Data
- A New Look at Ghost Normalization