Bonsai -- Diverse and Shallow Trees for Extreme Multi-label Classification

Abstract

Extreme multi-label classification (XMC) refers to supervised multi-label learning involving hundreds of thousand or even millions of labels. In this paper, we develop a suite of algorithms, called Bonsai, which generalizes the notion of label representation in XMC, and partitions the labels in the representation space to learn shallow trees. We show three concrete realizations of this label representation space including : (i) the input space which is spanned by the input features, (ii) the output space spanned by label vectors based on their co-occurrence with other labels, and (iii) the joint space by combining the input and output representations. Furthermore, the constraint-free multi-way partitions learnt iteratively in these spaces lead to shallow trees. By combining the effect of shallow trees and generalized label representation, Bonsai achieves the best of both worlds - fast training which is comparable to state-of-the-art tree-based methods in XMC, and much better prediction accuracy, particularly on tail-labels. On a benchmark Amazon-3M dataset with 3 million labels, \bonsai outperforms a state-of-the-art one-vs-rest method in terms of prediction accuracy, while being approximately 200 times faster to train. The code for Bonsai is available at \url{https://github.com/xmc-aalto/bonsai}

References in corpus (3)

• Distributed Representations of Words and Phrases and their Compositionality
• Large-scale Multi-label Learning with Missing Labels
• Gradient tree boosting with random output projections for multi-label classification and multi-output regression

Cited by in corpus (21)

• The Emerging Trends of Multi-Label Learning
• DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documents
• DECAF: Deep Extreme Classification with Label Features
• ECLARE: Extreme Classification with Label Graph Correlations
• On Missing Labels, Long-tails and Propensities in Extreme Multi-label Classification
• Fast Multi-Resolution Transformer Fine-tuning for Extreme Multi-label Text Classification
• Pretrained Generalized Autoregressive Model with Adaptive Probabilistic Label Clusters for Extreme Multi-label Text Classification
• Multi-modal Extreme Classification
• Probabilistic Label Trees for Extreme Multi-label Classification
• LdSM: Logarithm-depth Streaming Multi-label Decision Trees
• Propensity-scored Probabilistic Label Trees
• Top-$k$ eXtreme Contextual Bandits with Arm Hierarchy
• Unbiased Loss Functions for Extreme Classification With Missing Labels
• Extreme Meta-Classification for Large-Scale Zero-Shot Retrieval
• Multi-Head Encoding for Extreme Label Classification
• Online probabilistic label trees
• IRLI: Iterative Re-partitioning for Learning to Index
• On-the-fly Global Embeddings Using Random Projections for Extreme Multi-label Classification
• Multi-granular Legal Topic Classification on Greek Legislation
• Extreme Multi-label Learning for Semantic Matching in Product Search
• On Riemannian Approach for Constrained Optimization Model in Extreme Classification Problems