Deep Neural Decision Trees

Abstract

Deep neural networks have been proven powerful at processing perceptual data, such as images and audio. However for tabular data, tree-based models are more popular. A nice property of tree-based models is their natural interpretability. In this work, we present Deep Neural Decision Trees (DNDT) -- tree models realised by neural networks. A DNDT is intrinsically interpretable, as it is a tree. Yet as it is also a neural network (NN), it can be easily implemented in NN toolkits, and trained with gradient descent rather than greedy splitting. We evaluate DNDT on several tabular datasets, verify its efficacy, and investigate similarities and differences between DNDT and vanilla decision trees. Interestingly, DNDT self-prunes at both split and feature-level.

presented at 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018), Stockholm, Sweden

References in corpus (5)

• Deep Learning in Neural Networks: An Overview
• Categorical Reparameterization with Gumbel-Softmax
• Towards A Rigorous Science of Interpretable Machine Learning
• Hierarchical Multiscale Recurrent Neural Networks
• Neural Decision Trees

Cited by in corpus (33)

• A Survey on the Explainability of Supervised Machine Learning
• TabTransformer: Tabular Data Modeling Using Contextual Embeddings
• A novel embedded min-max approach for feature selection in nonlinear support vector machine classification
• ExplAIn: Explanatory Artificial Intelligence for Diabetic Retinopathy Diagnosis
• Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data
• Scalable Rule-Based Representation Learning for Interpretable Classification
• Learning Interpretable Rules for Scalable Data Representation and Classification
• Boost then Convolve: Gradient Boosting Meets Graph Neural Networks
• A Gradient Boosting Approach for Training Convolutional and Deep Neural Networks
• DNF-Net: A Neural Architecture for Tabular Data
• Oblique Decision Trees from Derivatives of ReLU Networks
• A Federated Learning Benchmark on Tabular Data: Comparing Tree-Based Models and Neural Networks
• A Framework for Democratizing AI
• Deep differentiable forest with sparse attention for the tabular data
• Cloud2Curve: Generation and Vectorization of Parametric Sketches
• Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration
• A Survey of Techniques All Classifiers Can Learn from Deep Networks: Models, Optimizations, and Regularization
• Muddling Label Regularization: Deep Learning for Tabular Datasets
• Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descent
• Stochastic Gradient Trees
• Enhancing Transformers with Gradient Boosted Decision Trees for NLI Fine-Tuning
• Gradient Boosted Decision Tree Neural Network
• Attention augmented differentiable forest for tabular data
• Dive into Decision Trees and Forests: A Theoretical Demonstration
• A short note on the decision tree based neural turing machine
• Learning Representations for Axis-Aligned Decision Forests through Input Perturbation
• Making Classical Machine Learning Pipelines Differentiable: A Neural Translation Approach
• Yet Another Representation of Binary Decision Trees: A Mathematical Demonstration
• A Nonparametric Ensemble Binary Classifier and its Statistical Properties
• Decision Machines: Congruent Decision Trees
• Proceedings of the First Workshop on Weakly Supervised Learning (WeaSuL)
• Convex Polytope Trees
• Efficient Click-Through Rate Prediction for Developing Countries via Tabular Learning