Yet Another Representation of Binary Decision Trees: A Mathematical Demonstration
arXiv:2101.07077
Abstract
A decision tree looks like a simple directed acyclic computational graph, where only the leaf nodes specify the output values and the non-terminals specify their tests or split conditions. From the numerical perspective, we express decision trees in the language of computational graph. We explicitly parameterize the test phase, traversal phase and prediction phase of decision trees based on the bitvectors of non-terminal nodes. As shown, the decision tree is a shallow binary network in some sense. Especially, we introduce the bitvector matrix to implement the tree traversal in numerical approach, where the core is to convert the logical `AND' operation to arithmetic operations. And we apply this numerical representation to extend and unify diverse decision trees in concept.
References in corpus (11)
- Categorical Reparameterization with Gumbel-Softmax
- From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
- Entity Embeddings of Categorical Variables
- Deep Neural Decision Trees
- A Regularized Framework for Sparse and Structured Neural Attention
- Efficient non-greedy optimization of decision trees
- Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data
- On Controllable Sparse Alternatives to Softmax
- Sigsoftmax: Reanalysis of the Softmax Bottleneck
- The Tree Ensemble Layer: Differentiability meets Conditional Computation
- The Power of Unbiased Recursive Partitioning: A Unifying View of CTree, MOB, and GUIDE