GradientBoost Notes

XGBoost
A Reader's Encyclopedia

An independent knowledge resource for the XGBoost algorithm. We don't replace the official docs — we complement them with concept explanations, mathematical derivations, annotated papers, and unbiased benchmarks. Maintained by open-source contributors. Not affiliated with the XGBoost project.

XGBoost (eXtreme Gradient Boosting) is one of the most widely used machine learning algorithms in production today. It powers winning solutions on Kaggle, drives recommendation systems at scale, and handles tabular data with unmatched speed and accuracy. This encyclopedia covers everything from foundational concepts to advanced optimization techniques, with fully worked mathematical derivations and reproducible Python code examples. Whether you're preparing for a data science interview, optimizing a production pipeline, or simply curious about how gradient boosting works under the hood — you'll find clear, rigorous, and practical explanations here.