[PYTHON] Continue to make stock price forecast AI for 10 hours a day 2nd month 2nd week

plan

I've done most of the easy things. From here, what can we do to improve the practical performance of the model? I currently have two ideas.

Add some information to the image in the dataset. For example, long-term trends and trading volumes.
Automatically learn in several layers and save only the best results to improve performance while sleeping.

I tried No. 1, but I will do No. 2 from tomorrow.

Question 1.

63.77% of the images in the dataset are labeled as "up" and this model may overfit to fire "up" in rapid succession. In practice, the purpose of this model and general deep learning is to minimize the loss function, and accuracy doesn't really matter in learning, so this doesn't seem to be a problem.

Reference. As an experiment, I tried to learn by setting the ratio of "up" to "down" in the data set to 1: 1 by removing the image labeled "up" appropriately. This result was obtained after 75 epochs.

This does not show the effect of dataset imbalances on the prediction accuracy of the model, but at least this result shows that the model outperforms the coin tossed bot.

Problem 2.

Due to the extreme simplification of the dataset and model, the only function of this model is to determine if the price is going up or down just two months later, taking into account price fluctuations during that period. Not. Therefore, realistic operation may be dangerous.

Consider practicality.

I've added trading volume to the dataset images to give the model maximum information about the financial markets, which doesn't seem to be a bad idea, but the problem is that the trading volume scale can vary from image to image. .. I think I need to learn a little more about Pyplot.

Let Python learn 2203 images from 2010.

About 60 epochs looks good, so set Epochs = 60 and try again.

Looks good.

When this model makes predictions using test data, there is a 74.64% chance that it will answer "up" and 66.4% of all test datasets will be labeled "up". If the model outputs "up" with a probability of 74.64% at random, the expected accuracy when tested with this test data is 58.07%, so this model with 63.7% accuracy is at least higher than the guesswork. It can be said that it is performance.

Conclusion

This model has become a so-called longer brain that thinks the market price will rise almost all the time, and it seems that it is not very practical. However, on the other hand, it seems that this model could be used to detect signs of recession. This is because when a model that tries to output "up" as much as possible outputs "down" as the answer, there may be something behind that judgment. We will now investigate the results and predictions of this model to see if that is possible.

Currently, this model and dataset have many problems, and I would like to solve each problem one by one and eventually create a practical AI. As an aside, if you let your computer learn, you will be surprised at the time it takes to process. I've read an article about quantum computers in Newton, but if the time required for this process is dramatically shortened, AI and quantum computers are a perfect match.

Plan from now on

Learn about pyplot because the way to add trading volume is so terrible
Learn the data of Lehman shock
Try it in the real market
Increase the amount of data by 10 times or more. Currently, we are using Toyota data from 40 representative American companies and Japanese companies, but I would like to extend this to all S & P 500 companies and Nikkei 225 companies. Or is it better to use only two ETFs as data? Anyway, I would like to make various trials and errors on the data to be used even after trying this. From now on, I would like to specify which data I learned.

see you next week.