[PYTHON] Regression analysis with NumPy

Regression analysis was mentioned earlier in The topic of linear regression.

Regression analysis with NumPy

Now, let's actually write the code and perform regression analysis. np.polyfit and np.polyval /reference/generated/numpy.polyval.html#numpy.polyval) can be used to perform regression analysis of two variables with an n-th order equation.

For details, it is better to refer to the document from the above link, but it is as follows.

np.polyfit (x, y, n): Regression analysis of two variables with the following equation
np.polyval (p, t): Substitute t for the polynomial represented by p and calculate the value

import numpy as np
import matplotlib.pyplot as plt

x = np.array([0.0, 1.0, 2.0, 3.0,  4.0,  5.0])
y = np.array([0.0, 0.8, 0.9, 0.1, -0.8, -1.0])
z = np.polyfit(x, y, 3)

p = np.poly1d(z)
p30 = np.poly1d(np.polyfit(x, y, 30))
xp = np.linspace(-2, 6, 100)

plt.plot(x, y, '.', xp, p(xp), '-', xp, p30(xp), '*')
plt.ylim(-2,2)
plt.show()
plt.savefig('image.png')

At this time In the case of a linear equation, p [0]: slope and p [1]: intercept. For N-th order equation, p [0] * t ** (N-1) + p [1] * t ** (N-2) + ... + p [N-2] * t + p [N-1] ] is.

Regarding the least squares method I mentioned before, the data strings {(x_1, y_1), (x_2, y_2), ... (x_n,, Determine the model that minimizes the residual sum of squares for y_n)}. At this time, it is assumed that the variance of the error distribution of the observed values is constant.

Multiple regression analysis

Regression analysis with m independent variables is [Multiple Regression Analysis](http://en.wikipedia.org/wiki/%E9%87%8D%E5%9B%9E%E5%B8%B0%E5% 88% 86% E6% 9E% 90).

z = ax + by + c

I want to find the formula.

x = [9.83, -9.97, -3.91, -3.94, -13.67, -14.04, 4.81, 7.65, 5.50, -3.34]
y = [-5.50, -13.53, -1.23, 6.07, 1.94, 2.79, -5.43, 15.57, 7.26, 1.34]
z = [635.99, 163.78, 86.94, 245.35, 1132.88, 1239.55, 214.01, 67.94, -1.48, 104.18]

import numpy as np
from scipy import linalg as LA

N = len(x)
G = np.array([x, y, np.ones(N)]).T
result = LA.solve(G.T.dot(G), G.T.dot(z))

print(result)

This [ -30.02043308 3.55275314 322.33397214] Can be obtained

z = -30.0x + 3.55y + 322

Is obtained, and this is the solution.

reference

2013 Mathematics Study Group Materials for Programmers http://nineties.github.io/math-seminar/