[PYTHON] You will be an engineer in 100 days ――Day 81 ――Programming ――About machine learning 6

Click here until yesterday

You will become an engineer in 100 days --Day 76 --Programming --About machine learning

You will become an engineer in 100 days-Day 70-Programming-About scraping

You will become an engineer in 100 days --Day 66 --Programming --About natural language processing

You become an engineer in 100 days --Day 63 --Programming --About probability 1

You will become an engineer in 100 days-Day 59-Programming-Algorithms

You will become an engineer in 100 days --- Day 53 --Git --About Git

You will become an engineer in 100 days --Day 42 --Cloud --About cloud services

You will become an engineer in 100 days --Day 36 --Database --About the database

You will be an engineer in 100 days --Day 24 --Python --Basics of Python language 1

You will become an engineer in 100 days --Day 18 --Javascript --JavaScript basics 1

You will become an engineer in 100 days --Day 14 --CSS --CSS Basics 1

You will become an engineer in 100 days --Day 6 --HTML --HTML basics 1

This time is a continuation of the story about machine learning.

About regression model

I will explain what you can do with machine learning for the first time, but what you can do with machine learning There are basically three.

・ Regression ・ Classification ・ Clustering

Roughly speaking, it becomes prediction, but the part of what to predict changes.

・ Regression: Predict numerical values ・ Classification: Predict categories ・ Clustering: Make it feel good

The regression model goes to predict the numbers.

The data used this time is the Boston house price data attached to scikit-learn.

The MEDV is the objective variable that you want to predict, and the others are the explanatory variables.

Data visualization

First, let's see what kind of data it is.

from sklearn.datasets import load_boston

#Data reading
boston = load_boston()
#Creating a data frame
boston_df = pd.DataFrame(data=boston.data,columns=boston.feature_names)
boston_df['MEDV'] = boston.target

#Data overview
print(boston_df.shape)
boston_df.head()

It contains numerical data.

Let's visualize to see the relationship between each column.

sns.pairplot(data=boston_df[list(boston_df.columns[0:6])+['MEDV']])
plt.show()

sns.pairplot(data=boston_df[list(boston_df.columns[6:13])+['MEDV']])
plt.show()

Let's also look at the correlation of each column.

boston_df.corr()

With the exception of some columns, the correlation between each column does not seem to be that high.

The regression model is that you want to rely on the value of one objective variable using these columns.

Creating a predictive model

** Data split **

First, split the data for training and testing. This time we will split at 6: 4.

from sklearn.model_selection import train_test_split

#6 for training and test data:Divided by 4
X = boston_df.drop('MEDV',axis=1)
Y = boston_df['MEDV']
x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=0.4, random_state=0)

** Modeling ** Next, create a forecast model.

Here, we will use the linear regression model to build a regression model with the objective variable MEDV and the explanatory variables CRIM ~ LSTAT.

Roughly speaking, Y = x1 * a + x2 * b + ... ϵ It is an image of making an expression like that.

ʻA and bare calledregression coefficients`, and each variable is It shows how much it contributes to the prediction of the objective variable.

ϵ is called residual, and represents the degree of deviation between each data and the expression. In the linear regression model, the sum of the residual squares of each data Find each coefficient by minimizing it.

The library used is called linear_model.

from sklearn import linear_model

#Learning with linear regression
model = linear_model.LinearRegression()
model.fit(x_train, y_train)

Modeling is done immediately by calling the library and doing fit.

** Accuracy verification **

In the accuracy verification of the regression model, we will look at how much the prediction and the actual measurement are different.

As a commonly used index Mean squared error (MSE) and Root mean squared error (RMSE) There is a R-squared value $ R ^ 2 $.

The `MSE is the average value of the sum of squares of the error, and if both the training data and the test data are small, it is judged that the performance of the model is good.

`RMSE is the square root of the mean squared error.

The R-squared value $ R ^ 2 $ takes 1 when MSE is 0, and the better the model performance, the closer to 1.

from sklearn.metrics import mean_squared_error

y_pred = model.predict(x_test)

print('MSE : {0} '.format(mean_squared_error(y_test, y_pred)))
print('RMSE : {0} '.format(np.sqrt(mean_squared_error(y_test, y_pred))))
print('R^2 : {0}'.format(model.score(x_test, y_test)))

MSE : 25.79036215070245 RMSE : 5.078421226198399 R^2 : 0.6882607142538019

By the way, looking at the accuracy, the value of RMSE is off by about 5.0. On average, there is an error of this deviation from the house price.

** Residual plot **

By the way, how much did the prediction model deviate? Let's visualize the residual.

#Plot the residuals
plt.scatter(y_pred, y_pred - y_test, c = 'red', marker = 's')
plt.xlabel('Predicted Values')
plt.ylabel('Residuals')

# y =Draw a straight line to 0
plt.hlines(y = 0, xmin = -10, xmax = 50, lw = 2, color = 'blue')
plt.xlim([10, 50])
plt.show()

By combining the test data and the prediction data, you can see how much the deviation is. Those that are out of alignment are quite out of alignment.

With this kind of feeling, create a model so that there is less deviation, select data, preprocess, adjust model parameter values, etc. We aim to improve accuracy with less error.

Summary

Today I explained how the regression model works. There are many other regression models.

First of all, let's start by saying what regression is, and suppress how to model and verify.

19 days until you become an engineer

Author information

Otsu py's HP: http://www.otupy.net/

Youtube： https://www.youtube.com/channel/UCaT7xpeq8n1G_HcJKKSOXMw

Twitter： https://twitter.com/otupython