Скачать или смотреть Gradient Boosting - Machine Learning

The name "Gradient Boosting" comes from the fact that the algorithm uses the gradient of the loss function to "boost" the performance of a weak learner.

The "boosting" part refers to the general class of algorithms that combine weak learners into a strong learner, by iteratively adding new models that focus on the misclassified samples from previous models.

In Gradient Boosting, each new model is trained to fit the residual errors of the previous model, weighted by the negative gradient of the loss function.

This results in a sequence of models that gradually improve the overall prediction accuracy by reducing the bias and variance of the final ensemble.

The term "gradient" refers to the gradient of the loss function that is being optimized. The loss function measures the difference between the predicted values of the model and the actual values of the target variable. The goal of Gradient Boosting is to iteratively improve the model's predictions by minimizing the loss function.

Implementations:
✅ XGBoost - Scalable and fast gradient boosting
✅ LightGBM - High-performance gradient boosting
✅ CatBoost - Handles categorical features efficiently
✅ GradientBoostingRegressor - Built-in sklearn package
✅ H2O - Scalable and distributed

Sunit Ghosh

#MachineLearning #GradientBoosting #GradientDescent