"Getting stock price time series data from k-db.com with Python" Program environment creation memo

In the Cloud9 development environment, install jupyter and run the stock price acquisition program from k-db.com in that environment.

Postscript 2018.2.22 It was said that k-db will end its service at the end of 2017.

I am studying deep learning and am thinking of a program to make stock price forecasts using stock data. However, there is almost no way to get the data of all TSE stocks in 5 minutes even if you search variously. And finally I found k-db.com. This site is for 4 days, but information on 5 minute bars of individual stocks is posted. However, it takes a lot of time to download and edit this one by one. And I found a program that automatically acquires stock price data in 5 minutes from this site. You can find it at the URL below.

Program to get stock price from k-db.com I used this program for "Getting time series data from k-db.com with Python". http://qiita.com/sawadybomb/items/03c3814268d3e2904e6c cloud9, python I'm new to python and it doesn't work very well.

environment

cloud9 (paid version) jupyter python 3.5.3

procedure

First, create an environment for anaconda and jupyter.

See here http://qiita.com/KenjiKitagawa/items/e0225d9be10ea3dbdafc

Download and deploy stock price acquisition program from k-db.com

https://github.com/sawadyrr5/pykdb Download the program from here to your PC and deploy it under the jupyter of cloud9.

Launch a file called setup.py

Execute setup of the downloaded file. Execute python setup.py install from the command line

When I started "tutorial_stocks.ipynb" under the example folder from jupyter, it worked properly.

However, depending on the brand, there was a part where the data was missing, and even when I looked at the acquisition source k-db.com, the data was still not included. The coping method is written below.

Program execution result NAN has no data

Example of countermeasures when part of the data becomes NAN

Since the original data of k-db.com is NAN, it is impossible to get the data. Originally, this data needs a proper numerical value to be used for deep learning of artificial intelligence, but since the number of NANs is small, I decided to input the numerical value by predicting the numerical value from the previous and next numerical values (linear numerical value).

Put df = df.interpolate (hoge) under df = obj.hoge in the sample program under the example directory. Example

df = obj.price(date_from=start, date_to=end, symbol=i_old, freq=stock_freq)   
df = df.interpolate(method='index') #Interpolate NAN data with front and back linear values

After complementation