[PYTHON] I tried to get started with Bitcoin Systre on the weekend
background
――When I was surfing the internet, I found a very helpful site about Systre. -Understand even in the humanities! How to start Bitcoin BOT automatic trading ――It was explained in a very easy-to-understand manner ――I feel like I can do it even if I have no experience with Systre. ――I was interested, so I decided to write a code on the weekend to check the feel. -Github This is the code I wrote this time. (Notebook format)
--By the way, before, Challenge to predict virtual currency by LSTM, it didn't work very well. ――This time, I wrote a simple algorithm using a moving average line without jumping to machine learning suddenly.
What you did
――We will try the effect by acquiring past data without actually trading. ――I wrote only the parts that seemed to be the basis of acquisition of past data and transaction simulation.
1. Get Bitcoin data from CryptWatch API
--Defined the following classes --Since it is difficult to see the JSON that can be obtained via API, we added a process to convert it to DataFrame format.
import json
import requests
import pandas as pd
import datetime
from pprint import pprint
import time
class PriceGetterCryptwatch():
#Omission
def get_price(self, periods, after=None, before=None, exchange="bitflyer", currency="btcjpy"):
"""
input
periods:Cycle 86400 for daily bars
after:Get chart after after with unixTime
before:Get chart before before with unixTime
exchange:Cryptocurrency exchange
currency:Currency pair
output
API results:JSON format
"""
url = "https://api.cryptowat.ch/markets/"+exchange+"/"+currency+"/ohlc?periods="+str(periods)
if after:
after=int(after)
url += "&after=" + str(after)
elif before:
before=int(before)
url += "&before=" + str(before)
print(url)
responce = requests.request("GET", url)
return responce.json()
def apidata_to_dataframe(self, data, periods):
"""
[ CloseTime , OpenPrice , HighPrice , LowPrice , ClosePrice , Volume]
[Date and time,Open price,High price,Low price,closing price,Volume]
"""
#Since the last column is unknown, delete it for the time being
df = pd.DataFrame(data["result"][str(periods)],columns=[ "CloseTime" , "OpenPrice" , "HighPrice" , "LowPrice" , "ClosePrice" , "Volume", "None"])
df = df.drop("None", axis=1)
#Convert Unix time to real time
df["CloseTime"] = df["CloseTime"].map(self.unixtime_to_datetime)
return df
--If you use it, you will get the following table
Saving past data
--Cryptwatch seems to be able to collect up to 6000 data from the latest. ――The reference also described how to save and accumulate past data. --Save in JSON format --I wanted to save it in csv with the appearance of DataFrame, so I rewrote it with pandas
function
- save_data --Simply save the acquired data in CSV
- update_data --Added newly acquired data that does not exist in the read csv. Create a new csv
def save_data(periods):
getter = PriceGetterCryptwatch()
date = getter.datetime_to_obj(2017,1,1)
date = getter.datetime_to_unixtime(date)
# designate start date
data = getter.get_price(periods, after=date)
df = getter.apidata_to_dataframe(data, periods)
start_time = (df["CloseTime"].iloc[0])
last_time = (df["CloseTime"].iloc[-1]) # unixtime
filename = str(start_time) + " - " + str(last_time) + "period " + str(periods)
filename += ".csv"
filename = filename.replace(":", "-")
df.to_csv(filename, index=False, float_format="%.6f")
def update_data(old_filename, periods):
df_old = pd.read_csv(old_filename)
old_size = df_old.shape[0]
# get new data
getter = PriceGetterCryptwatch()
date = getter.datetime_to_obj(2018,1,1)
date = getter.datetime_to_unixtime(date)
# designate start date
data = getter.get_price(periods,after=date)
df = getter.apidata_to_dataframe(data, periods)
df_temp = pd.concat([df_old, df], axis=0)
df_temp["CloseTime"] = pd.to_datetime(df_temp["CloseTime"])
df_save = df_temp.drop_duplicates(subset="CloseTime")
df_save = df_save.sort_values("CloseTime")
df_save = df_save.reset_index(drop=True)
new_size = df_save.shape[0] - old_size
if new_size > 0:
print("NewData : {}".format(new_size))
start_time = df_save["CloseTime"].iloc[0]
last_time = df_save["CloseTime"].iloc[-1] # unixtime
filename = str(start_time) + " - " + str(last_time) + "period " + str(periods)
filename += ".csv"
filename = filename.replace(":", "-")
dir_name = os.path.dirname(old_filename)
filename = os.path.join(dir_name, filename)
df_save.to_csv(filename, index=False, float_format="%.6f")
else:
print("NewData None")
--I got a little stuck in deleting and combining duplicate data in update_data --Combine old and new data with concat --After that, if you do not convert the date to Datetime again, drop_duplicates may not be judged as duplicate. --Even when saving to csv, if the number of digits of the float is not the same, it may not be judged as duplicate.
--Execute --When acquired in 1 minute
PERIOD_1MIN = 60
FILENAME_OLD_1MIN = "./Data/2019-11-06 12-45-00 - 2019-11-11 09-03-00period 60.csv"
update_data(FILENAME_OLD_1MIN, PERIOD_1MIN)
--Result --It seems that 258 new data have been added. Update is quick because it is 1 minute
https://api.cryptowat.ch/markets/bitflyer/btcjpy/ohlc?periods=60&after=1514732400
NewData : 258
--Acquired data .csv --More than 6000 data have been accumulated ――We will utilize it as learning data in the future.
CloseTime,OpenPrice,HighPrice,LowPrice,ClosePrice,Volume
2019-11-06 12:45:00,1013343,1013343,1013343,1013343,0.090000
2019-11-06 12:46:00,1013343,1013622,1012862,1013091,2.730759
2019-11-06 12:47:00,1013375,1013839,1013345,1013345,1.170000
...
2019-11-11 13:26:00,981670,982509,981670,982509,0.778839
2019-11-11 13:27:00,982489,982489,982003,982003,0.205953
2019-11-11 13:28:00,982052,982052,982052,982052,0.010000
2. Transaction simulation
--Try to make a transaction based on the data obtained by hitting the API
BitCoin price plot (4 hour bar and moving average (12h, 20h))
--Obtained BTC-JPY data from January 1, 2018 to the present --The vertical axis is the price, and the horizontal axis is the time. --The 12-hour moving average and the 20-hour moving average are also plotted. ――When you look at it again, the crash in 2018 is terrifying.
――I will expand it --If orange (short-term average line) and green (long-term average line) cross, it will be regarded as a trend change and you will buy and sell.
Implementation of trading algorithm
--Created a basic trading parts class and inherited it to create a trade algo ――The moment the short-term line crosses the long-term line and exceeds BUY --The moment the short-term line crosses the long-term line and falls below SELL
--First, set your own JPY and BitCoin and repeat buying and selling based on Argo. ――It is a gambler specification that puts all your JPY and BitCoin assets into each transaction.
class TradeBase():
#Omission
def __init__(self, jpy_asset, bitcoin_asset, period):
self.jpy_asset = jpy_asset
self.bitcoin_asset = bitcoin_asset
self.period = period
self.df = None
def buy_coin(self, bitcoin_price, jpy_asset, bitcoin_asset, jpy_buy_amount=None):
"""
For backtesting
"""
if jpy_buy_amount == None:
jpy_buy_amount = jpy_asset
bitcoin_buy_amount = jpy_buy_amount / bitcoin_price
new_jpy_asset = jpy_asset-jpy_buy_amount
new_bitcoin_asset = bitcoin_asset+bitcoin_buy_amount
return new_jpy_asset, new_bitcoin_asset
def sell_coin(self, bitcoin_price, jpy_asset, bitcoin_asset, bitcoin_sell_amount=None):
"""
For backtesting
"""
if bitcoin_sell_amount== None:
bitcoin_sell_amount = bitcoin_asset
jpy_sell_amount = bitcoin_price * bitcoin_sell_amount
new_jpy_asset = jpy_asset + jpy_sell_amount
new_bitcoin_asset = bitcoin_asset-bitcoin_sell_amount
return new_jpy_asset, new_bitcoin_asset
class TradeWithMovingAverage(TradeBase):
"""
When buying and selling only at the intersection of two types of moving averages
"""
def __init__(self, jpy_asset, bitcoin_asset, period, period_ma_type1, period_ma_type2):
super().__init__(jpy_asset, bitcoin_asset, period)
self.period_ma_type1 = period_ma_type1
self.period_ma_type2 = period_ma_type2
self.rolling_win_type1 = int(period_ma_type1 / period)
self.rolling_win_type2 = int(period_ma_type2 / period)
# for trade func
self.prev_mean_type1 = None
self.prev_mean_type2 = None
self.jpy_asset_result = []
self.data = None
def judge_sell_or_buy(self, new_mean_fast, prev_mean_fast, new_mean_slow, prev_mean_slow):
"""
Judgment of buying and selling by looking at the moving average line
"""
if prev_mean_fast > prev_mean_slow:
if new_mean_fast < new_mean_slow:
return "SELL"
else:
return "NOTHING"
elif prev_mean_fast < prev_mean_slow:
if new_mean_fast > new_mean_slow:
return "BUY"
else:
return "NOTHING"
else:
return "NOTHING"
def trade(self, init_var=True):
if self.data is None:
print(" Data is None")
return
if init_var:
self.prev_mean_type1 = None
self.prev_mean_type2 = None
self.jpy_asset_result = []
mean_type1 = None
mean_type2 = None
last_sell_jpyasset = None
for i, value in enumerate(self.data):
# Average
if i > self.rolling_win_type1:
mean_type1 = self.data.iloc[int(i-self.rolling_win_type1):i].mean()
if i > self.rolling_win_type2:
mean_type2 = self.data.iloc[int(i-self.rolling_win_type2):i].mean()
if mean_type1 is None or mean_type2 is None:
self.jpy_asset_result.append(self.jpy_asset)
continue
# Trade
bitcoin_price = value
if self.prev_mean_type1 and self.prev_mean_type2:
judge = self.judge_sell_or_buy(mean_type1, self.prev_mean_type1,
mean_type2, self.prev_mean_type2)
if judge == "SELL":
if self.bitcoin_asset > 0:
self.jpy_asset, self.bitcoin_asset = self.sell_coin(bitcoin_price, self.jpy_asset, self.bitcoin_asset)
last_sell_jpyasset = self.jpy_asset
elif judge == "BUY":
if self.jpy_asset > 0:
self.jpy_asset, self.bitcoin_asset = self.buy_coin(bitcoin_price, self.jpy_asset, self.bitcoin_asset, jpy_buy_amount=self.jpy_asset)
else:
pass
self.prev_mean_type1 = mean_type1
self.prev_mean_type2 = mean_type2
self.jpy_asset_result.append(self.jpy_asset)
self.last_sell_jpyasset = last_sell_jpyasset
print("Result: Jpy{:.1f} bitcoin{:.4f} (LAST JPY: {})".format(self.jpy_asset, self.bitcoin_asset, self.last_sell_jpyasset))
return self.jpy_asset, self.bitcoin_asset
Execution result
--Actually do this --Start trading from 10,000 Japanese Yen and 0 BTC ――Based on 4-hour data
jpy = 10000
bitcoin = 0
period = 60*60*4 #4 hours
trader = TradeWithMovingAverage(jpy_asset=jpy, bitcoin_asset=bitcoin, period=period,
period_ma_type1=period*3, period_ma_type2=period*5)
#Set of data(Manual)
trader.df = df
trader.data = df["ClosePrice"]
trader.trade()
trader.show_trade_result()
Asset transition (Japanese yen)
――It's more than tripled from 10,000 yen at the starting point, and it seems to be working unexpectedly. ――Assets have not decreased much even after the crash in the first half of 2018 --Argo has successfully contributed to the mid-2019 surge ――However, if there is no time for a surge, there seems to be no sign of increasing assets so much.
--I tried using it for the latest (2019/9 ~ 2019/11) data --Starting on 2019/9 with 10000 on hand --Not very effective. It seems that we have to wait for the next surge ――If you include the fee, you will lose it
3. Impressions
――It was clear to me that mechanical trading seems to be more stable without bias than humans do. ――I'm glad I got a sense of building a trading algorithm. ――I think we will make improvements that incorporate other indicators. ――It seems that there is still a lot of anxiety and it will take time to use it in practice. --Deep Learning, I would like to incorporate reinforcement learning into the trading judgment algorithm. ――Thank you for reading.
Recommended Posts