A Comparative Analysis of XGBoost

Abstract

XGBoost is a scalable ensemble technique based on gradient boosting that has demonstrated to be a reliable and efficient machine learning challenge solver. This work proposes a practical analysis of how this novel technique works in terms of training speed, generalization performance and parameter setup. In addition, a comprehensive comparison between XGBoost, random forests and gradient boosting has been performed using carefully tuned models as well as using the default settings. The results of this comparison may indicate that XGBoost is not necessarily the best choice under all circumstances. Finally an extensive analysis of XGBoost parametrization tuning process is carried out.

References in corpus (3)

• XGBoost: A Scalable Tree Boosting System
• 3FGL Demographics Outside the Galactic Plane using Supervised Machine Learning: Pulsar and Dark Matter Subhalo Interpretations
• KiDS-SQuaD II: Machine learning selection of bright extragalactic objects to search for new gravitationally lensed quasars

Cited by in corpus (29)

• Searches for the BSM scenarios at the LHC using decision tree based machine learning algorithms: A comparative study and review of Random Forest, Adaboost, XGboost and LightGBM frameworks
• Data-driven fluid mechanics of wind farms: A review
• Finding defects in glasses through machine learning
• Sequential Training of Neural Networks with Gradient Boosting
• Condensed Gradient Boosting
• Machine Learning methods to estimate observational properties of galaxy clusters in large volume cosmological N-body simulations
• The Three Hundred project: A Machine Learning method to infer clusters of galaxies mass radial profiles from mock Sunyaev-Zel'dovich maps
• Predicting electronic stopping powers using stacking ensemble machine learning method
• A Gradient Boosting Approach for Training Convolutional and Deep Neural Networks
• An Interpretable Boosting-based Predictive Model for Transformation Temperatures of Shape Memory Alloys
• Forecasting COVID-19 spreading trough an ensemble of classical and machine learning models: Spain's case study
• A Deep Features-Based Approach Using Modified ResNet50 and Gradient Boosting for Visual Sentiments Classification
• Robust-Multi-Task Gradient Boosting
• Machine learning approach for mapping the stable orbits around planets
• On Gradient Boosted Decision Trees and Neural Rankers: A Case-Study on Short-Video Recommendations at ShareChat
• The MIT Supercloud Workload Classification Challenge
• Machine Learning Classification of COSMOS2020 Galaxies: Quiescent vs. Star-Forming
• ML-Based Bidding Price Prediction for Pay-As-Bid Ancillary Services Markets: A Use Case in the German Control Reserve Market
• Vectorization of Gradient Boosting of Decision Trees Prediction in the CatBoost Library for RISC-V Processors
• Advanced Detection of Source Code Clones via an Ensemble of Unsupervised Similarity Measures
• Movement- and Traffic-based User Identification in Commercial Virtual Reality Applications: Threats and Opportunities
• Histogram-Based Federated XGBoost using Minimal Variance Sampling for Federated Tabular Data
• Functional Decomposition and Estimation of Irreversibility in Time Series via Machine Learning
• Optimizing BioTac Simulation for Realistic Tactile Perception
• Robust multi-task boosting using clustering and local ensembling
• Interpretable classification of wiki-review streams
• COSMOS2025: A Machine Learning Census of Massive Quiescent Galaxies at $2.5<\text{z}<5$
• Deep learning approaches show promise for predicting childhood malnutrition: A comparative study with traditional machine learning methods using survey data
• A gradient boosting and broadband approach to finding Lyman-α emitting galaxies beyond narrowband surveys