[PYTHON] I tried to organize the evaluation indexes used in machine learning (regression model)

• Articles sent by data scientists from the manufacturing industry
• This time, the evaluation indexes used in the regression model are organized.

Introduction

This time, I tried to organize the evaluation index (regression model) of machine learning. We also summarized the points when using each evaluation index.

1. Coefficient of determination

The coefficient of determination $ R ^ 2 $ evaluates the goodness of the estimated regression model. The formula is as follows.

\begin{eqnarray}
R^2 = 1 - \sum \frac{(y_i - \hat{y_i})^2}{(y_i - \bar{y})^2}
\end{eqnarray}

$ y_i $: Measured value of $ i $ th sample $ \ hat {y_i} $: Predicted value of $ i $ th sample $ \ bar {y} $: Average of measured values

The python code is below.

#Library for score calculation
from sklearn.metrics import r2_score

#true is the true value (measured value), pred is the predicted value
true = [1.0, 1.5, 2.0, 1.2, 1.8]
pred = [0.8, 1.5, 1.8, 1.3, 2.0]

#Calculate the value of the coefficient of determination R2
r2_score(true, pred)

2. MAE MAE (Mean Absolute Error) evaluates the magnitude of the error in the predicted value. The formula is as follows.

\begin{eqnarray}
MAE = \frac{1}{N} \sum|y_i - \hat{y_i}|
\end{eqnarray}

$ y_i $: Measured value of $ i $ th sample $ \ hat {y_i} $: Predicted value of $ i $ th sample $ N $: Number of samples

The python code is below.

#Library for score calculation
from sklearn.metrics import mean_absolute_error

#true is the true value (measured value), pred is the predicted value
true = [1.0, 1.5, 2.0, 1.2, 1.8]
pred = [0.8, 1.5, 1.8, 1.3, 2.0]

#Calculate the value of MAE
mean_absolute_error(true, pred)

3. MSE In MSE (Mean Squared Error), large errors are emphasized and evaluated. The formula is as follows.

\begin{eqnarray}
MSE = \frac{1}{N} \sum(y_i - \hat{y_i})^2
\end{eqnarray}

$ y_i $: Measured value of $ i $ th sample $ \ hat {y_i} $: Predicted value of $ i $ th sample $ N $: Number of samples

The python code is below.

#Library for score calculation
from sklearn.metrics import mean_squared_error

#true is the true value (measured value), pred is the predicted value
true = [1.0, 1.5, 2.0, 1.2, 1.8]
pred = [0.8, 1.5, 1.8, 1.3, 2.0]

#Calculate the value of MSE
mean_squared_error(true, pred)

4. RMSE In RMSE (Root Mean Squared Error), large errors are emphasized and evaluated. The formula is as follows.

\begin{eqnarray}
RMSE = \sqrt{\frac{1}{N} \sum(y_i - \hat{y_i})^2}
\end{eqnarray}

$ y_i $: Measured value of $ i $ th sample $ \ hat {y_i} $: Predicted value of $ i $ th sample $ N $: Number of samples

The python code is below.

#Library for score calculation
import numpy as np
from sklearn.metrics import mean_squared_error

#true is the true value (measured value), pred is the predicted value
true = [1.0, 1.5, 2.0, 1.2, 1.8]
pred = [0.8, 1.5, 1.8, 1.3, 2.0]

#Calculate RMSE value
np.sqrt(mean_squared_error(true, pred))

5. MAPE MAPE (Mean Absolute Percentage Error) evaluates the magnitude of the prediction error per measured value. The formula is as follows.

\begin{eqnarray}
MAPE = \frac{1}{N} \sum|\frac{y_i - \hat{y_i}}{y_i}|
\end{eqnarray}

$ y_i $: Measured value of $ i $ th sample $ \ hat {y_i} $: Predicted value of $ i $ th sample $ N $: Number of samples

The python code is below.

#Library for score calculation
import numpy as np

#true is the true value (measured value), pred is the predicted value
true = np.array([1.0, 1.5, 2.0, 1.2, 1.8])
pred = np.array([0.8, 1.5, 1.8, 1.3, 2.0])

#Calculate the value of MAPE
np.mean(np.abs((true - pred) / true)) * 100

6. SMAPE SMAPE (Symmetric Mean Absolute Percentage Error) evaluates the magnitude of the prediction error per measured value. The formula is as follows.

\begin{eqnarray}
SMAPE = \frac{100}{N} \sum|\frac{2(y_i - \hat{y_i})}{(|y_i|+|\hat{y_i}|)}|
\end{eqnarray}

$ y_i $: Measured value of $ i $ th sample $ \ hat {y_i} $: Predicted value of $ i $ th sample $ N $: Number of samples

The python code is below.

#Library for score calculation
import numpy as np

#true is the true value (measured value), pred is the predicted value
true = np.array([1.0, 1.5, 2.0, 1.2, 1.8])
pred = np.array([0.8, 1.5, 1.8, 1.3, 2.0])

#Calculate SMAPE value
100/len(true) * np.sum(2 * np.abs(pred - true) / (np.abs(pred) + np.abs(true)))

Script summary

Organize the scripts so far.

#Library for score calculation
import numpy as np
import pandas as pd
from sklearn.metrics import r2_score
from sklearn.metrics import mean_absolute_error
from sklearn.metrics import mean_squared_error

def calculate_scores(true, pred):
    """Calculate all metrics

    Parameters
    ----------
    true : np.array
Measured value
    pred : np.array
Predicted value

    Returns
    -------
    scores : pd.DataFrame
Results of summarizing each evaluation index
    """
    scores = pd.DataFrame({'R2': r2_score(true, pred),
                          'MAE': mean_absolute_error(true, pred),
                          'MSE': mean_squared_error(true, pred),
                          'RMSE': np.sqrt(mean_squared_error(true, pred)),
                          'MAPE': mape(true, pred),
                          'SMAPE': smape(true, pred)},
                          index = ['scores'])
    return scores

def mape(true, pred):  
    """Calculate MAPE

    Parameters
    ----------
    true : np.array
Measured value
    pred : np.array
Predicted value

    Returns
    -------
    np.array :mape calculation result
    """
    return np.mean(np.abs((true - pred) / true)) * 100

#SMAPE calculation
def smape(true, pred):
    """Calculate SMAPE

    Parameters
    ----------
    true : np.array
Measured value
    pred : np.array
Predicted value

    Returns
    -------
    np.array :Calculation of smape
    """
    return 100/len(true) * np.sum(2 * np.abs(pred - true) / (np.abs(pred) + np.abs(true)))

def main():
    true = np.array([1.0, 1.5, 2.0, 1.2, 1.8])
    pred = np.array([0.8, 1.5, 1.8, 1.3, 2.0])

    scores = calculate_scores(true, pred)

    print(scores)

if __name__ == "__main__":
    main()

finally

Thank you for reading to the end. If you have a request for correction, we would appreciate it if you could contact us.