table of contents

1. Result
2. About the model to use
3. How to evaluate the model
4. Try the model
5. Parameter tuning
6. Submit to Kaggle
7. Summary

History

1. Result

From the result, the score went up a little to "0.77511". The result is the top 58% (as of December 29, 2019). I would like to see the flow until resubmission.

2. About the model to use

Last time, I used "Linear SVC" according to scikit-learn algorithm sheet. I first learned machine learning [this book](https://www.amazon.co.jp/Python-%E6%A9%9F%E6%A2%B0%E5%AD%A6%E7%BF % 92% E3% 83% 97% E3% 83% AD% E3% 82% B0% E3% 83% A9% E3% 83% 9F% E3% 83% B3% E3% 82% B0-% E9% 81% 94% E4% BA% BA% E3% 83% 87% E3% 83% BC% E3% 82% BF% E3% 82% B5% E3% 82% A4% E3% 82% A8% E3% 83% B3% E3% 83% 86% E3% 82% A3% E3% 82% B9% E3% 83% 88% E3% 81% AB% E3% 82% 88% E3% 82% 8B% E7% 90% 86% E8% AB% 96% E3% 81% A8% E5% AE% 9F% E8% B7% B5-impress-gear / dp / 4295003379 / ref = dp_ob_title_bk) takes up the following model of scikit-learn on the classification problem. I am. ・ Sklearn.svm.LinearSVC ・ Sklearn.svm.SVC ・ Sklearn.ensemble.RandomForestClassifier ・ Sklearn.linear_model.LogisticRegression ・ Sklearn.linear_model.SGDClassifier

This time, I would like to try the above model.

3. How to evaluate the model

The procedure for evaluating the model is as follows.

1. Learn using training data
2. Predict using test data
3. Check if the predicted result is correct

Kaggle's Titanic has training data [train.csv](data with unknown results) and test data [test.csv](data with unknown results). If you use test.csv to "predict" 2 and "confirm" 3 each time, you have to commit and submit the result, which is inefficient. Training data [train.csv] with known results can be evaluated efficiently by dividing it into training data and test data. scikit-learn provides a function "train_test_split" that splits into training data and test data.

from sklearn.model_selection import train_test_split

######################################
#Separate training data and test data
# Split training data and test data
######################################
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.3, random_state=1, shuffle=True)

The image is as follows. If test_size = 0.3, the training data and test data will be divided by "7: 3".

〇Data before division

〇 Training data after division

〇 Test data after division

Next is learning and prediction.

The scikit-learn model provides a method "fit" for learning and a method "score" for evaluating predictions. They are "fit" and "score".

from sklearn.svm import LinearSVC
model = LinearSVC(random_state=1)

######################################
#learn
# training
######################################
model.fit(x_train, y_train)

######################################
#Evaluate the predicted results
# Evaluate predicted results
######################################
score = model.score(x_test, y_test)

“Fit” is learning. "Score" predicts the result with "x_test", matches the result with "y_test", and returns the correct answer rate. In the above case, the score would be "0.753731343283582". The result is a 75% correct answer rate.

4. Try the model

You can evaluate the performance of your model by experimenting with different models and comparing their scores. Try the model in "2. About the model to use".

The overall code is below.

Preparation

import numpy 
import pandas

# train.load csv
# Load train.csv
df = pandas.read_csv('/kaggle/input/titanic/train.csv')

##############################
#Data preprocessing
#Extract the required items
# Data preprocessing 
# Extract necessary items
##############################
# 'Survived', 'Pclass', 'Sex', 'Fare'To extract
# Extract 'Survived', 'Pclass', 'Age', 'Fare'
df = df[['Survived', 'Pclass', 'Sex', 'Fare']]

##############################
#Data preprocessing
#Quantify the label (name)
# Data preprocessing 
# Digitize labels
##############################
from sklearn.preprocessing import LabelEncoder
#Quantify gender using Label Encoder
# Digitize gender using LabelEncoder
encoder_sex = LabelEncoder()
df['Sex'] = encoder_sex.fit_transform(df['Sex'].values)

##############################
#Data preprocessing
#Standardize numbers
# Data preprocessing
# Standardize numbers
##############################
from sklearn.preprocessing import StandardScaler

#Standardization
# Standardize numbers
standard = StandardScaler()
df_std = pandas.DataFrame(standard.fit_transform(df[['Pclass', 'Fare']]), columns=['Pclass', 'Fare'])

#Standardize Fare
# Standardize Fare
df['Pclass'] = df_std['Pclass']
df['Fare'] = df_std['Fare']

from sklearn.model_selection import train_test_split

x = df.drop(columns='Survived')
y = df[['Survived']]

Training data, test data creation

#######################################
#Separate training data and test data
# Split training data and test data
#######################################
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.3, random_state=1, shuffle=True)
y_train = numpy.ravel(y_train)
y_test = numpy.ravel(y_test)

Model evaluation

#######################################
#Evaluate the model
# Evaluate the model
#######################################
from sklearn.svm import LinearSVC
model = LinearSVC(random_state=1)
model.fit(x_train, y_train)
score = model.score(x_test, y_test)
score

By replacing the definition part of the model in "Model evaluation", you can evaluate with various models. Try the model described in "2. About the model to use". The result is as follows.

The result is that Random Forest is the best. Next, let's adjust the parameters of the random forest model.

5. Parameter tuning

Use the grid search (GridSearchCV) in scikit-learn to adjust the parameters. Grid search evaluates the specified parameters in all patterns and finds the optimum combination of parameters. However, since all patterns are evaluated, the more parameters you have, the longer the process will take. Check the Random Forest Documentation (https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html) and decide to adjust the following parameters:

You can perform grid search by replacing "model evaluation" with the following "grid search".

Grid search

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV
###############################################
#Try LogisticRegression parameters with grid search
# Tuning LogisticRegression parameters with grid search
###############################################
pipe_svc = RandomForestClassifier(random_state=1)

param_grid = {'criterion':['gini','entropy'],
			  'n_estimators':[25, 100, 500, 1000, 2000],
			  'min_samples_split':[0.5, 2,4,10],
			  'min_samples_leaf':[1,2,4,10],
			  'bootstrap':[True, False]
			  }

grid = GridSearchCV(estimator=RandomForestClassifier(random_state=1), param_grid=param_grid)
grid = grid.fit(x_train, y_train)

print(grid.best_score_)
print(grid.best_params_)

The result is as follows. In my environment, it took about 10 minutes to execute the grid search.

Grid search results

0.8105939004815409
{'bootstrap': False, 'criterion': 'entropy', 'min_samples_leaf': 10, 'min_samples_split': 2, 'n_estimators': 100}

6. Submit to Kaggle

Let's specify the parameters tuned by grid search, learn and predict. Rewrite the code of "Training data, test data creation" and "Grid search" to the following to perform learning and prediction.

Learning, anticipation

##############################
#Model building
# Model building
##############################
from sklearn.ensemble import RandomForestClassifier

#Generate a model
# Generate a model
model = RandomForestClassifier(n_estimators=100, \
                               criterion='entropy', \
                               min_samples_split=2, \
                               min_samples_leaf=10, \
                               bootstrap=False, \
                               random_state=1)

##############################
#Learning
# Trainig
##############################
y = numpy.ravel(y)
model.fit(x, y)

# test.Convert csv
# convert test.csv
##############################
# test.load csv
# Load test.csv
df_test = pandas.read_csv('/kaggle/input/titanic/test.csv')

#Convert Fare Nan
# Convert Fare Nan to 0 
df_test = df_test.fillna({'Fare':0})

# 'PassengerId'To extract(To combine with the result)
# Extract 'PassengerId'(To combine with the result)
df_test_index = df_test[['PassengerId']]

# 'Pclass', 'Sex', 'Fare'To extract
# Extract 'Pclass', 'Sex', 'Fare'
df_test = df_test[['Pclass', 'Sex', 'Fare']]

#Standardization
# Standardize
df_test_std = pandas.DataFrame(standard.transform(df_test[['Pclass', 'Fare']]), columns=['Pclass', 'Fare'])
df_test['Pclass'] = df_test_std['Pclass']
df_test['Fare'] = df_test_std['Fare']

#Label encoding
# Label Encoding
df_test ['Sex'] = encoder_sex.transform(df_test ['Sex'].values)

##############################
#Predict results
# Predict results
##############################
x_test = df_test.values
y_test = model.predict(x_test)

#Combine the result with the DataFrame of the PassengerId
# Combine the data frame of PassengerId and the result
df_output = pandas.concat([df_test_index, pandas.DataFrame(y_test, columns=['Survived'])], axis=1)

# result.Write csv to current directory
# Write result.csv to the current directory
df_output.to_csv('result.csv', index=False)

Write the above in a Kaggle environment. Run "Run All" and verify that result.csv is created.

Submit by selecting "Commit" ⇒ "Open Version" ⇒ "Submit to Competition".

The score is now "0.77511".

7. Summary

This time, I was able to raise the score a little by comparing 5 types of models and tuning the parameters. Next time would like to find a more suitable model from various models of scikit-learn.

History

2019/12/29 First edition released 2020/01/01 Add next link 2020/01/03 Source comment correction