[PYTHON] Rebuilding order / position book using oandapy V20
1.First of all
I think that many people use only price (+ volume) when creating investment bots. But these are not the only data available. For example, the status of orders and the status of open positions can be considered.
This time, I took out order and position data (order book, position book) from oanda of Forex exchange and created a tool to process it, so I will share it.
2. What is an order book / position book?
An order book is a chart that shows how many unfilled orders are for each rate (a board in a stock). An open position shows how many positions you currently have at what rate. For details, please jump to the oanda site from the link and check.
（ https://www.oanda.jp/lab-education/oanda_lab/oanda_rab/open_position/ ）
3. About oanda books
With over $ 500,000 in transactions within the last month, oanda will update both your orderbook and positionbook once every 5 minutes. This frequency does not change whether you view it on the web or get it with the api.
If you just create an account, the update frequency will drop to once every 20 minutes. However, if you do not trade frequently, I think that the update frequency is sufficient.
We will not explain how to open an account or obtain an api key here.
4. Body
You can see that the open orders you see on the oanda site are spread out by a certain width (every 0.05 yen for dollar yen), centered on the rate at the time when the snapshot of the open order was created.
But the data obtained by the api is not that way. The order book contains a wide range of rate information, including information that is impossible in reality, such as 1 dollar = 0 yen to 1 dollar = 200 yen. Also, the rate at the heart of open orders fluctuates constantly. Therefore, even if you try to analyze the obtained data as it is, you will not know how far each rate is from the rate when the open order was created.
Therefore, this time, we have prepared a code that can collect and process data so that you can intuitively grasp the central rate and how far it is from it, as shown below. Please rewrite the accountID and access_token to your own.
There seems to be no data before January 20, 2017. Also, it takes about 20 to 30 minutes to get the book every hour.
def fetch_orderbook(instrument="USD_JPY", levels = 5, timeframe="15m"):
'''
instrument :Currency pair name
levels :How many up (down) to get from the central rate, >0
timeframe :How many minutes to get the book, "m":minute, "H":hour, "D":day, >="5m"
ex) "1H":Get every hour, "2D":Get daily
'''
accountID = "×××-×××-×××-×××"
access_token ="××××××××××××"
api = API(access_token=access_token, environment="live")
if timeframe[-1]=="m":
diff = 60 * int(timeframe[:-1])
elif timeframe[-1]=="H":
diff = 60 * 60 * int(timeframe[:-1])
elif timeframe[-1]=="D":
diff = 60 * 60 * 24 * int(timeframe[:-1])
timestamp = datetime.datetime(year=2017, month=1, day=20)
now = datetime.datetime.utcnow()
processed_book, book_timestamps = [], []
counter = 0
start = datetime.datetime.utcnow()
while True:
try:
#Finish when you finish getting until now
if now < timestamp:
break
#Prevention of excessive access to the server and log output
counter+=1
if counter%100==0:
print(timestamp, datetime.datetime.utcnow()-start)
start = datetime.datetime.utcnow()
counter=0
time.sleep(1)
#Get orderbook
r=instruments.InstrumentsOrderBook(instrument=instrument, params={"time":timestamp.strftime("%Y-%m-%dT%H:%M:00Z")})
api.request(r)
book_created_rate, buckets, book_created_time = float(r.response["orderBook"]["price"]), r.response["orderBook"]["buckets"], r.response["orderBook"]["time"]
#Get an index close to the rate at the time of book creation
bucket_rate = [round(float(b["price"]), 2) for b in buckets]
upper_idx = np.searchsorted(bucket_rate, book_created_rate)
lower_idx = upper_idx - 1
#Added levels separated from current price
row_book = [book_created_rate]
for i in range(levels):
upper, lower = buckets[upper_idx + i], buckets[lower_idx - i]
bucket = [upper["price"], upper['longCountPercent'], upper['shortCountPercent'],
lower["price"], lower['longCountPercent'], lower['shortCountPercent']]
bucket = [float(num) for num in bucket]
row_book.extend(bucket)
processed_book.append(row_book)
book_timestamps.append(book_created_time)
except:
#Saturday and Sunday
pass
timestamp+=datetime.timedelta(seconds = diff)
#colum name
column_names = ["rate"]
for i in range(levels):
column_names.extend(["price_+{}".format(i+1), "OrderLongPercent_+{}".format(i+1), "OrderShortPercent_+{}".format(i+1),
"price_-{}".format(i+1), "OrderLongPercent_-{}".format(i+1), "OrderShortPercent_-{}".format(i+1)])
#Data combination
orderbook = pd.DataFrame(processed_book, columns=column_names, index=book_timestamps)
return orderbook
The same applies to the position book
def fetch_positionbook(instrument="USD_JPY", levels = 5, timeframe="15m"):
'''
instrument :Currency pair name
levels :How many up (down) to get from the central rate, >0
timeframe :How many minutes to get the book, "m":minute, "H":hour, "D":day, >="5m"
ex) "1H":Get every hour, "2D":Get daily
'''
accountID = "×××-×××-×××-×××"
access_token ="××××××××××××"
api = API(access_token=access_token, environment="live")
if timeframe[-1]=="m":
diff = 60 * int(timeframe[:-1])
elif timeframe[-1]=="H":
diff = 60 * 60 * int(timeframe[:-1])
elif timeframe[-1]=="D":
diff = 60 * 60 * 24 * int(timeframe[:-1])
timestamp = datetime.datetime(year=2017, month=1, day=20)
now = datetime.datetime.utcnow()
processed_book, book_timestamps = [], []
counter = 0
start = datetime.datetime.utcnow()
while True:
try:
#Finish when you finish getting until now
if now < timestamp:
break
#Prevention of excessive access to the server and log output
counter+=1
if counter%100==0:
print(timestamp, datetime.datetime.utcnow()-start)
start = datetime.datetime.utcnow()
counter=0
time.sleep(1)
#Get positionbook
r=instruments.InstrumentsPositionBook(instrument=instrument, params={"time":timestamp.strftime("%Y-%m-%dT%H:%M:00Z")})
api.request(r)
book_created_rate, buckets, book_created_time = float(r.response["positionBook"]["price"]), r.response["positionBook"]["buckets"], r.response["positionBook"]["time"]
#Get an index close to the rate at the time of book creation
bucket_rate = [round(float(b["price"]), 2) for b in buckets]
upper_idx = np.searchsorted(bucket_rate, book_created_rate)
lower_idx = upper_idx - 1
#Added levels separated from current price
row_book = [book_created_rate]
for i in range(levels):
upper, lower = buckets[upper_idx + i], buckets[lower_idx - i]
bucket = [upper["price"], upper['longCountPercent'], upper['shortCountPercent'],
lower["price"], lower['longCountPercent'], lower['shortCountPercent']]
bucket = [float(num) for num in bucket]
row_book.extend(bucket)
processed_book.append(row_book)
book_timestamps.append(book_created_time)
except:
#Saturday and Sunday
pass
timestamp+=datetime.timedelta(seconds = diff)
#colum name
column_names = ["rate"]
for i in range(levels):
column_names.extend(["price_+{}".format(i+1), "PositionLongPercent_+{}".format(i+1), "PositionShortPercent_+{}".format(i+1),
"price_-{}".format(i+1), "PositionLongPercent_-{}".format(i+1), "PositionShortPercent_-{}".format(i+1)])
#Data combination
positionbook = pd.DataFrame(processed_book, columns=column_names, index=book_timestamps)
return positionbook
5. Finally
I'd like to introduce it, but I think it's difficult to build a bot using only this data. I was able to win (without the commission), but I couldn't seem to win with the commission. Maybe it works well when combined with a time anomaly ...?
Recommended Posts