[PYTHON] Generalized linear model (GLM) and neural network are the same (1)

It sounds a bit rough, but it's the same thing as a generalized linear model (GLM) and a neural network multi-layer perceptron. In the scattered data, I'm going to draw a convincing line. I think there are many people who do both fields, but I think it's easier to understand at once.

In each of the statistical modeling and discriminant functions, ・ How about drawing a straight line in any data with a linear model (LM)? → Let's generalize (GLM) so that non-linear lines can be drawn.

• Of course, generalization also has the meaning of extending the distribution of response variables from the normal distribution. -Simple perceptrons can only classify data that can be linearly discriminated (boundaries can be drawn by straight lines). → Create a hidden layer (intermediate layer) and classify it with a complicated discriminant function. I expanded the feeling.

From the perspective of statistical modeling: Generalized Linear Models (GLM)

For Linear Model and Generalized Linear Model, The materials for the statistical solution study session are summarized here, which is quite easy to understand. " Statistical Study Group Materials-Day 2 Generalized Linear Models-: Logics of Blue "

So, if you write a linear model, a simple regression model or a multiple regression model in general,

{ \displaystyle
y_i = \alpha + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 … + \beta_i x_i + \varepsilon_i
}

When written in a matrix,

\begin{equation}

\left(
    \begin{array}{c}
      y_1 \\
      y_2 \\
・\\
・\\
      y_n
    \end{array} 

\right)

=

\left(

 \begin{array}{ccc}
      1 & x_{11} & x_{12} \\
      1 & x_{12} & x_{22} \\
･&･&･\\
･&･&･\\
      1 & x_{n1} & x_{n2}
    \end{array}
\right)

\left(
   \begin{array}{c}
     \alpha \\
     \beta_1 \\
･\\
･\\
     \beta_n 
    \end{array}
\right)

+

\left(
   \begin{array}{c}
     \varepsilon_1 \\
     \varepsilon_2 \\
･\\
･\\
     \varepsilon_n 
    \end{array}
\right)


\end{equation}

further

y = B X + \epsilon

Can also be written. This time, for the sake of simplicity, the explanation variate is set to 2.

y = \beta_1 x_1 + \beta_2 x_2 = BX

Think about.

Next, in order to remove the constraint of assuming only a straight line and a normal distribution from the above equation, Use the link function on the left side of the regression equation. Assuming a Bernoulli distribution, using the logit link function

\log \frac{y}{1-y} = \beta_1 x_1 + \beta_2 x_2

The left side is the link function and the right side is the linear predictor. When this formula is converted by the inverse function,

  y = \frac{1}{1+exp(-(\beta_1 x_1 + \beta_2 x_2))} = \frac{1}{1+exp(-BX)}　　(1)

Can be obtained.

In statistical modeling, after this, parameters are used using a likelihood function, etc.

\beta_1  \beta_2
```To build a model by estimating=Draw a line in the data.

 It's getting longer, so I'll continue in the next article.