[PYTHON] Try Bitcoin Price Forecasting with Deep Learning

Free study of summer vacation-Dream of unearned income-

I took a Certificate at Coursera, so it's a way to spend the summer vacation trying to make something like machine learning or deep learning. There are great people in the world, and there are already tiredly tried approaches, and I will try them with the thinking power that I have studied.

In the future, I would like to make a highly accurate model, make a dream unearned income, and pay taxes.

Introduction

It took me a day to study.

Environment

How to install TensorFlow that even beginners of machine learning can do immediately Many people are already doing it, so please imitate it. The thing to be careful about is to do it with Python3, it is good to specify the version to be used with pyenv etc.

I think it can be done in about 10 minutes.

Library

Keras

Keras is a high-level neural network library written in Python that can be run on TensorFlow, CNTK, and Theano.

I was wondering if Deep Learning should have TensorFlow, but of course there are many frameworks that wrap it, and Keras is one of them. You can choose to use theano/tensorflow as documented.

After all, both are for dealing with tensors, and I don't really understand the difference, but at first I thought it wasn't that important and skipped it.

numpy Needless to say, a convenient python numerical calculation

pandas A convenient guy who handles time series data

data = pandas.read_csv('./csv/bitcoin_log_1month.csv')
data = data.sort_values(by='date')
data = data.reset_index(drop=True)

It is convenient to load csv like this. In addition to loading, it has the ability to sort and index.

Scikit-learn It provides a convenient module for machine learning. Since it is premised on processing for numpy.ndarray, it will be executed for this. There are also normalization modules.

tflearn It's a library that makes TensorFlow use like Scikit-learn. It's built into TensorFlow, so you can build neural networks and define models.

matplotlib Everyone loves matlab. It is used to visualize the created graph and save it as an image. Since an error occurs when importing, modify matplotlibrc partly by referring to the link below.

What to do if matplotlib.pyplot cannot be used when building an environment with pyenv and virtualenv

design

What should I do first? Let's take a look at a lot of helpful links. After all, what I want to do is to make something that can solve the binary classification of "whether tomorrow's closing price will rise or fall". By calling a reference link, I was able to convey the desire to generate unearned income for everyone. Ah yeah.

reference

-Stock Price Forecast with TensorFlow (LSTM) ~ Stock Forecast Part 1 ~ -Forecast the number of airline passengers next month with RNN: Let's implement from LSTM to GRU with TFLearn -Try to predict the exchange rate (FX) with TensorFlow (deep learning)

Let's understand

After all, ML and DL are quite different, so what is handled is very different, so I organized it.

Required flow

1. Securing time series data

There are various places where bitcoin time series data can be obtained via API, but I wanted to do it investing ) To DL. You can take the annual closing price for several years. If it is not JPY, there are many other options.

2. Data set formatting

We will normalize and standardize the necessary data. At this time it is in sklearn [processing package](http://qiita.com/nazoking@github/items/d6ac1948ee138d73fef1#431-%E6%A8%99%E6%BA%96%E5%8C%96%E5%B9%B3%E5% 9D% 87% E9% 99% A4% E5% 8E% BB% E3% 81% A8% E5% 88% 86% E6% 95% A3% E3% 81% AE% E3% 82% B9% E3% 82% With B1% E3% 83% BC% E3% 83% AA% E3% 83% B3% E3% 82% B0) you can easily do the following:

data['close'] = preprocessing.scale(data['close'])

3. Selection of model and error to use

The required error and model differ depending on the data used, so even if you do not have much knowledge here, you can verify by replacing the arguments. I think you can really study properly and get meaningful results.

Principle of easy deep learning

LSTM (Long Short-Term Memory) is a powerful model that eliminates the drawbacks of RNNs and can learn long-term time series data.

There are various models such as GRU and LSTN, but there are various models, although I will not mention the features.

The rest is error

The meaning of various errors (RMSE, MAE, etc.) There are various types such as MAPE and RMSE. Since it is an evaluation index, it evaluates how far the measured value and the predicted value are when the model is changed or the amount of data is added.

4. Preparation of training data and test data

Select how much data you want to learn and how much data you want to compare with. Training: I felt that I often set test = 8: 2.

5. Prediction based on these

A model is created using the constructed neural network, fitting is performed based on the model, and prediction is performed.

6. Evaluation

After that, the results are graphed, visualized and evaluated. With TensorBoard, you can easily visualize models and errors.

I learned from the references that I should be able to implement this flow roughly.

Implementation

# -*- coding: utf-8 -*-
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import tflearn

from sklearn import preprocessing
from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.layers.recurrent import LSTM

class Prediction :

    def __init__(self):
        self.dataset = None
        #Values to calculate
        self.model = None
        self.train_predict = None
        self.test_predict = None

        #Data set parameter settings
        self.steps_of_history = 3
        self.steps_in_future = 1
        self.csv_path = './csv/bitcoin_log.csv'

    def load_dataset(self):
        #Data preparation
        dataframe = pd.read_csv(self.csv_path,
                usecols=['closing price'],
                engine='python').sort_values('closing price', ascending=True)
        self.dataset = dataframe.values
        self.dataset = self.dataset.astype('float32')

        #Standardization
        self.dataset -= np.min(np.abs(self.dataset))
        self.dataset /= np.max(np.abs(self.dataset))


    def create_dataset(self):
        X, Y = [], []
        for i in range(0, len(self.dataset) - self.steps_of_history, self.steps_in_future):
            X.append(self.dataset[i:i + self.steps_of_history])
            Y.append(self.dataset[i + self.steps_of_history])

        X = np.reshape(np.array(X), [-1, self.steps_of_history, 1])
        Y = np.reshape(np.array(Y), [-1, 1])
        return X, Y

    def setup(self):
        self.load_dataset()
        X, Y = self.create_dataset()

        # Build neural network
        net = tflearn.input_data(shape=[None, self.steps_of_history, 1])

        #GRU because LSTM takes time
        # http://dhero.hatenablog.com/entry/2016/12/02/%E6%9C%80%E5%BC%B1SE%E3%81%A7%E3%82%82%E6%A9%9F%E6%A2%B0%E5%AD%A6%E7%BF%92%E3%81%A7%E3%81%8A%E9%87%91%E3%81%8C%E7%A8%BC%E3%81%8E%E3%81%9F%E3%81%84%E3%80%905%E6%97%A5%E7%9B%AE%E3%83%BBTFLearn%E3%81%A8
        net = tflearn.gru(net, n_units=6)
        net = tflearn.fully_connected(net, 1, activation='linear')

        #Regression settings
        #Measured by Adam method
        # http://qiita.com/TomokIshii/items/f355d8e87d23ee8e0c7a
        #Mean as an index of prediction accuracy in time series analysis_I'm using square
        #mape seems to be common
        # categorical_crossentropy
        # mean_square :Root mean square
        net = tflearn.regression(net, optimizer='adam', learning_rate=0.001,
                loss='mean_square')

        # Define model
        self.model = tflearn.DNN(net, tensorboard_verbose=0)

        #This time 80%The training dataset, 20%Is treated as a test data set.
        pos = round(len(X) * (1 - 0.2))
        trainX, trainY = X[:pos], Y[:pos]
        testX, testY   = X[pos:], Y[pos:]
        
        return trainX, trainY, testX

    def executePredict(self, trainX, trainY, testX):
        # Start training (apply gradient descent algorithm)
        self.model.fit(trainX, trainY, validation_set=0.1, show_metric=True, batch_size=1, n_epoch=150, run_id='btc')

        # predict
        self.train_predict = self.model.predict(trainX)
        self.test_predict = self.model.predict(testX)

    def showResult(self):
        # plot train data
        train_predict_plot = np.empty_like(self.dataset)
        train_predict_plot[:, :] = np.nan
        train_predict_plot[self.steps_of_history:len(self.train_predict) + self.steps_of_history, :] = \
                self.train_predict

        # plot test dat
        test_predict_plot = np.empty_like(self.dataset)
        test_predict_plot[:, :] = np.nan
        test_predict_plot[len(self.train_predict) + self.steps_of_history:len(self.dataset), :] = \
                self.test_predict

        # plot show res
        plt.figure(figsize=(8, 8))
        plt.title('History={} Future={}'.format(self.steps_of_history, self.steps_in_future))
        plt.plot(self.dataset, label="actual", color="k")
        plt.plot(train_predict_plot, label="train", color="r")
        plt.plot(test_predict_plot, label="test", color="b")
        plt.savefig('result.png')
        plt.show()


if __name__ == "__main__":

    prediction = Prediction()
    trainX, trainY, testX = prediction.setup()
    prediction.executePredict(trainX, trainY, testX)
    prediction.showResult()



I referred to the site I referred to very much, but it took about an hour to implement and understand.

Enter a brief code description.

load_dataset

The data used this time is like this. Read this and use it as a data set. Since the order of data is reversed, sort etc.

"Date","closing price","Open price","High price","Low price","The day before ratio"
"September 03, 2017","523714.1875","499204.7813","585203.1250","499204.7813","4.91"
"September 02, 2017","499204.7813","542277.3125","585203.1250","498504.5000","-7.94"

setup Build a neural network. GRU is one of the DL methods. There are various things such as LSTM and RNN, but this time I chose it with an emphasis on speed.

net = tflearn.input_data(shape=[None, self.steps_of_history, 1])
net = tflearn.gru(net, n_units=6)
net = tflearn.fully_connected(net, 1, activation='linear')

Selection of optimization method / miscalculation This time, the optimization is Adam method and the error is mean_square (RMSE (Root Mean Square Error)). / rmsemae) is used. Since methods such as MAPE are not prepared by default, it seems necessary to partially rewrite the library so that it can be used.

net = tflearn.regression(net, optimizer='adam', learning_rate=0.001,
                loss='mean_square')

executePredict

Select and predict training / test data.

#This time 80%The training dataset, 20%Is treated as a test data set.
pos = round(len(X) * (1 - 0.2))
trainX, trainY = X[:pos], Y[:pos]
testX, testY   = X[pos:], Y[pos:]

# Start training (apply gradient descent algorithm)
self.model.fit(trainX, trainY, validation_set=0.1, show_metric=True, batch_size=1, n_epoch=150, run_id='btc')

# predict
self.train_predict = self.model.predict(trainX)
self.test_predict = self.model.predict(testX)

See the result

Black is the actual data, red is the training data, and blue is the predicted value based on the test data.

passenger.png

Here is the result of TensorBoard.

スクリーンショット 2017-09-06 21.40.18.png

It's like Humuu. Loss / Validation is strange for the time being. From here, we will do our best to investigate parameters that are likely to reduce the error while changing the number of Epoch, model and step, training data ratio and amount of data, number of GRU layers, etc.

By the way, I tried to predict using LSTM with sklearn without using tflearn as follows.

    model.add(LSTM(self.hidden_neurons, \
              batch_input_shape=(None, self.length_of_sequences, self.in_out_neurons), \
              return_sequences=False))
    model.add(Dense(self.in_out_neurons))
    model.add(Activation("linear"))
    model.compile(loss="mape", optimizer="adam")

スクリーンショット 2017-09-03 14.51.28.png

mape is not in tflearn, but is it in sklearn ...? ?? Very difficult.

Summary

I made something that works for the time being, but the summer vacation is over and the rest will be done at a later date. Now that I can move it, I will learn more about the verification method and improve the accuracy.

after

Visual studio Code for Mac was very good. I thought the implementation of python was Pycham's choice, but it's very good.

スクリーンショット 2017-09-06 22.00.07.png

Recommended Posts

Try Bitcoin Price Forecasting with Deep Learning
Try deep learning with TensorFlow
Try Deep Learning with FPGA
Try Deep Learning with FPGA-Select Cucumbers
Try deep learning with TensorFlow Part 2
Forecasting Snack Sales with Deep Learning
Try with Chainer Deep Q Learning --Launch
Try deep learning of genomics with Kipoi
Deep Kernel Learning with Pyro
Try machine learning with Kaggle
Generate Pokemon with Deep Learning
Try to build a deep learning / neural network with scratch
Cat breed identification with deep learning
Predict Bitcoin price changes with Prophet
Reinforcement learning 13 Try Mountain_car with ChainerRL.
Solve three-dimensional PDEs with deep learning.
Try machine learning with scikit-learn SVM
Try mining Bitcoin with Python's hashlib
Check squat forms with deep learning
Categorize news articles with deep learning
Try Common Representation Learning with chainer
Make people smile with Deep Learning
Introduction to Deep Learning (2) --Try your own nonlinear regression with Chainer-
Deep Learning
Classify anime faces with deep learning with Chainer
Deep learning / Deep learning from scratch 2-Try moving GRU
Stock price forecast using deep learning (TensorFlow)
Sentiment analysis of tweets with deep learning
Reinforcement learning 11 Try OpenAI acrobot with ChainerRL.
The story of doing deep learning with TPU
Deep Learning Memorandum
Start Deep learning
99.78% accuracy with deep learning by recognizing handwritten hiragana
Stock Price Forecast Using Deep Learning (TensorFlow) -Part 2-
Python Deep Learning
Deep learning × Python
Stock price forecast using deep learning [Data acquisition]
A story about predicting exchange rates with Deep Learning
Try to predict forex (FX) with non-deep machine learning
Extract music features with Deep Learning and predict tags
Classify anime faces by sequel / deep learning with Keras
Now, let's try face recognition with Chainer (learning phase)
First Deep Learning ~ Struggle ~
Learning Python with ChemTHEATER 03
Learning Python with ChemTHEATER 05-1
Python: Deep Learning Practices
Deep learning / activation functions
Deep Learning from scratch
Deep learning 1 Practice of deep learning
Deep learning / cross entropy
First Deep Learning ~ Preparation ~
First Deep Learning ~ Solution ~
[AI] Deep Metric Learning
Learning Python with ChemTHEATER 02
I tried deep learning
Learning Python with ChemTHEATER 01
Try SNN with BindsNET
Python: Deep Learning Tuning
Deep learning large-scale technology
Try regression with TensorFlow
Deep learning / softmax function