[PYTHON] Supervised learning 2 Hyperparameters and tuning (1)

Aidemy　2020/9/25

Introduction

Hello, it is Yope! I am a liberal arts student, but I was interested in the possibilities of AI, so I went to the AI-specialized school "Aidemy" to study. I would like to share the knowledge gained here with you, and I am summarizing it on Qiita. I am very happy that many people have read the previous summary article. Thank you! This is the second post of supervised learning. Nice to meet you.

• This article is a summary of what you learned in "Aidemy" "in your own words". It may contain mistakes and misunderstandings. Please note.

What to learn this time ・ About hyperparameters -Logistic regression, linear SVM, non-linear SVM hyperparameters

Hyperparameters

What are hyperparameters?

-Hyperparameters are __ "areas (parameters) that people have to adjust" __ in the machine learning model. The parameters differ depending on the type of model. (Looked at individually in the following sections) -Artificially adjusting hyperparameters is called tuning. -Parameters can be tuned when building a model. (The method will be described later)

Logistic regression hyperparameters

Parameter C

-One of the hyperparameters of logistic regression is a parameter called C. (Initial value 1.0) ・ C is an index of __ "how faithfully the boundary is drawn on the teacher data" __. That is, if the value of C is high, a boundary line is drawn so that all teacher data can be classified, but it can be said that overfitting is likely to occur. -Specify as __model = LogisticRegression (C = 1.0) __.

Parameter penalty

-The parameter penalty is __ "when the model becomes too complicated, the feature amount (L1) or the total weight (L2) is reduced so that it can be generalized appropriately". -Specify as penalty = L1 in the argument of Logistic Regression.

Parameter multi_class

-The parameter multi_class indicates __ "behavior when classifying multiple classes" __. -In logistic regression, in binary classification, it is specified as ovr, which is the behavior of "belonging / not belonging to class". In the multinomial classification, it is specified as multinomial, which is the behavior of "how much it may belong to".

Parameter random_state

-Random_state is __ "Random number seed that determines the data processing order" __, and by fixing this, the data processing order (= learning result) is also fixed.

Linear SVM hyperparameters

Each parameter

-C: The content and usage are the same. However, changes in the value of C have a greater effect on the accuracy rate than logistic regression. -__ Penalty __: Same as logistic regression. -Multi_class: In linear SVM, ovr and crammer_singer can be specified. Basically, ovr gives better results. In addition, it is not necessary to set this parameter at the time of binary classification. -Random_state: In linear SVM, this value also affects when determining the support vector, and the result may change slightly.

Non-linear SVM hyperparameters

Parameter C

-C: The content and usage are the same. However, the penalty is done by adjusting the value of C.

Parameter kernel

-Kernel is a __parameter that specifies the kernel function used for the __ "Convert non-linear classification to linear classification" operation, which is the key to processing non-linear SVMs. You can specify "rbf", "poly", "linear", "sigmoid", and "precomputed", but "rbf", which is the default value and has a high accuracy rate, is often used. -"Rbf" and "poly" are stereoscopic projections, "linear" is the same behavior as linear SVM (so it is rarely used), "sigmoid" is the same behavior as the logistic regression model, and "precomputed" is already formatted Used at the time.

Parameter decision_function_shape

-Decision_function_shape shows the behavior when deciding which class the data belongs to, like multi_class. You can specify "ovo" and "ovr". -Ovo is a method of combining two classes into one and binarizing each data for all cases. ovr classifies (directly) whether data belongs to that class. Since ovo has a large amount of calculation, the operation tends to be heavy.

Parameter random_state

-Random_state: For non-linear SVC, a random number generator must be created separately.

import numpy as np
from sklearn.svm import SVC
#Creating a random number generator
a = np.random.RandomState()
model = SVC(random_state = a)

Summary

-Parameters that must be artificially adjusted in the model are called hyperparameters, and the hyperparameters that should be set differ depending on the model type. -Logistic regression and linear SVM hyperparameters include __ "C" "penalty" "multi_class" "random_state" __. -Nonlinear SVM hyperparameters include __ "C", "kernel", "desicion_function_shape", and "random_state" __.

This time is over. Thank you for reading this far.