[PYTHON] Neural network implementation (NumPy only)

Introduction

This article is a continuation of Machine Learning ③ Introduction / Implementation of Activation Function. This article is an article about implementing a neural network using only NumPy. Some parts of the link below do not overlap with the explanation, so please read them as well. Machine learning ① Basics of Perceptron basics Machine learning ② Perceptron activation function Machine learning ③ Introduction and implementation of activation function

reference

References: O'REILLY JAPAN Deep Learning from scratch Articles so far: Machine learning ① Basics of Perceptron basics Machine learning ② Perceptron activation function Machine learning ③ Introduction and implementation of activation function

Premise

Implement matrix multiplication in Python

First, you need to know about the calculation of matrix multiplication when building a neural network. As expected, it would be a pain to write the calculation method of the matrix in this article, so please add O'REILLY JAPAN Deep Learning from scratch. I think you should read it. Only the calculation method is not described, so it is enough to have it googled.

Matrix product implementation

Take the following as an example.

Figure 4-1

Let's actually implement the above in Python.

3-1step_func.py

import numpy as np

A = np.array([[1, 2], [3, 4], [5, 6]])

#.Get shape with shape
print(A.shape)

B = np.array([[1, 2, 3, 4],[5, 6, 7, 8]])
print(B.shape)

#np.Get matrix product with dot
C = np.dot(A, B)

print(C.shape)
print(C)

Execution result

(3, 2)
(2, 4)
(3, 4)
[[11 14 17 20]
 [23 30 37 44]
 [35 46 57 68]]

You can see that the result is the same as in Figure 4-1. We will use this implementation method to help build neural networks.

Neural network and matrix product

If you express the neural network as a matrix product, you can calculate it all at once. It is shown below.

Figure 4-2

This is an example in which the input is represented by a 1 and 2 matrix, the weight is represented by a 2 and 3 dimensional matrix, and the output is represented by 1 and 3.

4-2TwoLayer_NeuralNetwork.py

import numpy as np

x = np.array([1, 2])
w = np.array([[1, 3, 5], [2, 4, 6]])

y = np.dot(x, w)
print(y)

Execution result

[ 5 11 17]

You can see that the output value is output.

Summary

I tried to express the neural network by matrix product. Next time, I will actually introduce the activation function by taking a three-layer neural network as an example.