Forward Thinking: Building Deep Random Forests

Abstract

The success of deep neural networks has inspired many to wonder whether other learners could benefit from deep, layered architectures. We present a general framework called forward thinking for deep learning that generalizes the architectural flexibility and sophistication of deep neural networks while also allowing for (i) different types of learning functions in the network, other than neurons, and (ii) the ability to adaptively deepen the network as needed to improve results. This is done by training one layer at a time, and once a layer is trained, the input data are mapped forward through the layer to create a new learning problem. The process is then repeated, transforming the data through multiple layers, one at a time, rendering a new dataset, which is expected to be better behaved, and on which a final output layer can achieve good performance. In the case where the neurons of deep neural nets are replaced with decision trees, we call the result a Forward Thinking Deep Random Forest (FTDRF). We demonstrate a proof of concept by applying FTDRF on the MNIST dataset. We also provide a general mathematical formulation that allows for other types of deep learning problems to be considered.

References in corpus (2)

• Scikit-learn: Machine Learning in Python
• Forward Thinking: Building and Training Neural Networks One Layer at a Time

Cited by in corpus (8)

• Polynomial Regression As an Alternative to Neural Nets
• Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data
• Forward Thinking: Building and Training Neural Networks One Layer at a Time
• Muddling Label Regularization: Deep Learning for Tabular Datasets
• Efficient Click-Through Rate Prediction for Developing Countries via Tabular Learning
• Growing Deep Forests Efficiently with Soft Routing and Learned Connectivity
• Discovering Drug-Drug and Drug-Disease Interactions Inducing Acute Kidney Injury Using Deep Rule Forests
• An Adaptive Weighted Deep Forest Classifier