[PYTHON] Machine learning ⑤ AdaBoost Summary
Summary of AdaBoost
What is AdaBoost?
AdaBoost is a machine learning model that attempts to create a strong discriminator by combining weak discriminators that are slightly more accurate than random. The flow of how to make First, apply a weak discriminator and increase the weight of those that have been misclassified. Then, the ones with the weight are given priority and classified. I repeat that.
It is easy to understand if you refer to the figure below. I also posted a Youtube link, so if you want to know more, please take a look.
Extracted from Alexander Ihler's youtube video
In the figure above, D1 first uses a weak classifier to classify and then D2 misclassifies'+' 1 and'-' 2 weights. Next, the three misclassified items are prioritized and classified again. Here, at the same time as increasing the weight, the weight of others that are correctly classified is decreasing. Furthermore, in D3, the weights of the'-'3 misclassified in D2 are increased and at the same time the weights of others are decreased. By the way, in AdaBoost's default code, the Decision Tree is used for classification by this weak discriminator.
Based on the weight of the repeated classification, we will make a strong discriminator.
Extracted from Alexander Ihler's youtube video
default code
python
from sklearn.ensemble import AdaBoostClassifier
AdaBoostClassifier(base_estimator=None, n_estimators=50, learning_rate=1.0, algorithm='SAMME.R', random_state=None)
A description of the Parameters in AdaBoost
- base_estimator
A machine learning model used as a weak discriminator. DecisionTreeClassifier is used for default, but other machine learning models can be adapted by specifying it.
- n_estimators
Specify how many times to repeat the classification using a weak discriminator. However, before that, if the strong discriminator becomes 100% accurate, it ends there.
The pros and cons of AdaBoost.
- good point
Since some weak classifiers are combined, it is easy to classify accurately.
--Bad point
Same as K nearest neighbors, it is also vulnerable to noise (different labels are mixed in the same place) and Outliers (outliers). Prone to overfitting.
Summary
The above is an overview of AdaBoost as far as I can tell. We will update it daily, so if you have something to add or fix, we would appreciate it if you could comment.
Recommended Posts