Adaptive Neural Trees

Abstract

Deep neural networks and decision trees operate on largely separate paradigms; typically, the former performs representation learning with pre-specified architectures, while the latter is characterised by learning hierarchies over pre-specified features with data-driven architectures. We unite the two via adaptive neural trees (ANTs) that incorporates representation learning into edges, routing functions and leaf nodes of a decision tree, along with a backpropagation-based training algorithm that adaptively grows the architecture from primitive modules (e.g., convolutional layers). We demonstrate that, whilst achieving competitive performance on classification and regression datasets, ANTs benefit from (i) lightweight inference via conditional computation, (ii) hierarchical separation of features useful to the task e.g. learning meaningful class associations, such as separating natural vs. man-made objects, and (iii) a mechanism to adapt the architecture to the size and complexity of the training dataset.

arXiv commentInternational Conference on Machine Learning 2019

Cited by in corpus (14)

• Dynamic Neural Networks: A Survey
• MurTree: Optimal Classification Trees via Dynamic Programming and Search
• Attention Convolutional Binary Neural Tree for Fine-Grained Visual Categorization
• Surprisal-Triggered Conditional Computation with Neural Networks
• Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descent
• Dive into Decision Trees and Forests: A Theoretical Demonstration
• Dynamic Slimmable Denoising Network
• Tree in Tree: from Decision Trees to Decision Graphs
• Interpretable Mixture Density Estimation by use of Differentiable Tree-module
• Weakly Supervised Recovery of Semantic Attributes
• Making CNNs Interpretable by Building Dynamic Sequential Decision Forests with Top-down Hierarchy Learning
• Learning Multi-Layered GBDT Via Back Propagation
• Dynamic Parameterized Network for CTR Prediction
• A Neural Tangent Kernel Perspective of Infinite Tree Ensembles