Continuing from Part 1, in the second part, I would like to actually refer to various APIs and operate the BOT service. The source code is operated at GitBucket, and the finished product is operated at here.
Since there are API usage restrictions to monitor each user's battle record and post it to Twitter if there is an update, it is necessary to control the process to be executed at intervals of seconds. Since Cron, which will be described later, can only be called at a minimum interval of 1 minute, each user's processing uses a mechanism called Task Queue provided by GAE. As an image of the Task Queue, the tasks are thrown into a container called Bucket, and the tasks are processed according to the settings in the order in which they were put first. If you increase the Bucket size, it will be parallel processing. The configuration file is queue.yaml.
queue.yaml
queue:
- name: tweet #Anything is fine
rate: 0.8/s #0 per second.Process at a pace of 8 times
bucket_size: 1 #Number to process in parallel
retry_parameters:
task_retry_limit: 0 #Number of retries when processing fails
Now, use add ()` `` to actually throw a task from Python into the Bucket set in` `` queue.yaml. Specify `name``` set in queue.yaml``` for `` `` queue_name appropriately. Specify the address of the process to be called in ```url appropriately`. You can also pass parameters.
launcher.py
from google.appengine.api.taskqueue import add
import webapp2
class modelTask(db.Model): #Queued tasks
resion = db.StringProperty()
summoner_name = db.StringProperty()
summoner_id = db.IntegerProperty()
latest_game = db.IntegerProperty()
access_key = db.StringProperty()
access_secret = db.StringProperty()
date_success = db.DateTimeProperty(auto_now_add=True)
date = db.DateTimeProperty(auto_now_add=True)
class mainHandler(webapp2.RequestHandler):
def get(self):
qs = modelTask.all().order('-date_success')
for q in qs: #Add all tasks to Queue
add(queue_name='tweet', url='/tweet', params={'qid': q.key().id()})
app = webapp2.WSGIApplication([ ('/launcher', mainHandler) ])
Next, implement the process specified in ```url appropriately` that is called when the turn comes around in the Task Queue. Called by the POST method from the Task Queue.
tweet.py
#! -*- coding: utf-8 -*-
from google.appengine.ext import db
from google.appengine.api.urlfetch import fetch
from django.utils.simplejson import loads
import webapp2, tweepy
from datetime import datetime
from laucher import modelTask
CONSUMER_KEY = '********************'
CONSUMER_SECRET = '**************************************'
RIOT_KEY = '***********************************'
class mainHandler(webapp2.RequestHandler):
def post(self):
getGame(long(self.request.get('qid')))
def getGame(qid):
q = modelTask().get_by_id(qid, parent=None)
#Call the RIOT API
result = fetch('https://prod.api.pvp.net/api/lol/'+q.resion+'/v1.3/game/by-summoner/'+str(q.summoner_id)+'/recent?api_key='+RIOT_KEY)
if result.status_code == 200:
#Set various values obtained from API
j = loads(result.content)['games'][0]
if j['stats']['win'] == True:
win = 'victory'
else:
win = 'defeat'
try:
kill = str(j['stats']['championsKilled'])
except:
kill = '0'
try:
death = str(j['stats']['numDeaths'])
except:
death = '0'
try:
assist = str(j['stats']['assists'])
except:
assist = '0'
game_type = j['subType']
#Post on Twitter if there is an update in the final battle time
if j['createDate'] > q.latest_game:
q.latest_game = j['createDate']
q.put()
if tweet(q.summoner_name+'Mr. latest'+game_type+'The battle record is'+kill+'kill'+death+'death'+assist+'Assist'+win+'is . http://tol.orfx.jp #Tweet_of_Legends', q.access_key, q.access_secret):
q.date_success = datetime.now()
q.put()
#Twitter post processing
def tweet(message, access_key, access_secret):
try:
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth_handler=auth, api_root='/1.1', secure=True)
api.update_status(status=message)
return True
except:
return False
app = webapp2.WSGIApplication([ ('/tweet', mainHandler) ])
I use Cron because I want to periodically call the process of adding the above task to the queue. The configuration file is cron.yaml. This time, I'll try to run it every 10 minutes from 1 am to 6 pm, and every 5 minutes otherwise.
cron.yaml
cron:
- description: tweet job
url: /launcher
schedule: every 10 minutes from 1:00 to 17:59
timezone: Asia/Tokyo #Don't forget the time zone when specifying the time
- description: tweet job
url: /launcher
schedule: every 5 minutes from 18:00 to 0:59
timezone: Asia/Tokyo
By the way, if you run it with this Cron (every 5 to 10 minutes) setting, the instance startup time is calculated in 15-minute increments, so Frontend instance startup time will definitely consume 24 hours or more. In addition to that, registration page processing etc. will be added, so I am worried about the 28 hours of the free frame (as of March 2014). So, let's take advantage of Backend. Backend is originally a mechanism for doing behind-the-scenes work such as batch processing and asynchronous processing, but this time it is simply used to earn instance startup time (9 hours). The configuration file is backend.yaml.
backend.yaml
backends:
- name: tweet #anything is fine
class: B1 #Minimize processing resources
options: dynamic #Make the instance non-resident
Specify `` `target``` to make the process Backend in Cron. This time, I will process it with Backend from 6 pm to 1 am.
cron:
- description: tweet job
url: /launcher
schedule: every 10 minutes from 1:00 to 17:59
timezone: Asia/Tokyo
- description: tweet job
url: /launcher
schedule: every 5 minutes from 18:00 to 0:59
timezone: Asia/Tokyo
target: tweet #backend.yaml name
TaskQueue can also be processed by Backend by specifying add (target =) `` `. Use `` `get_backend ()` `` to see if you are running with Backend. Also, when Backend is started, `` `/ _ah / start is forcibly called. Since it is called first, you can describe the process at startup, but this time we will describe the empty process.
launcher.py
from google.appengine.api.taskqueue import add
from google.appengine.api.backends import get_backend
import webapp2
class modelTask(db.Model):
resion = db.StringProperty()
summoner_name = db.StringProperty()
summoner_id = db.IntegerProperty()
latest_game = db.IntegerProperty()
access_key = db.StringProperty()
access_secret = db.StringProperty()
date_success = db.DateTimeProperty(auto_now_add=True)
date = db.DateTimeProperty(auto_now_add=True)
class mainHandler(webapp2.RequestHandler):
def get(self):
qs = modelTask.all().order('-date_success')
target = get_backend()
if target is None: #If it is started by Backend, TaskQueue will be the same.
for q in qs:
add(queue_name='tweet', url='/tweet', params={'qid': q.key().id()})
else:
for q in qs:
add(queue_name='tweet', url='/tweet', params={'qid': q.key().id()}, target='tweet')
class startHandler(webapp2.RequestHandler): #Forced to be called when Backend starts
def get(self):
return
app = webapp2.WSGIApplication([ ('/launcher', mainHandler), ('/_ah/start', startHandler) ])
Memcache can be used to significantly reduce resource consumption for processes that frequently browse the data store but rarely update it. In RIOT API game-v1.3, Champion is returned by ID, so it is necessary to convert the ID to a name based on the information of champion-v1.1. Since you can't hit the API frequently, the information acquired by champion-v1.1 is stored once in the data store. Then copy the information stored in Memcache. Use `memcache.add (key, value)` to add data to Memcache.
champion.py
from google.appengine.api import memcache
from google.appengine.ext import db
from google.appengine.api.urlfetch import fetch
from django.utils.simplejson import loads
import webapp2
RIOT_KEY = '***********************************'
class modelChampion(db.Model): #Champion information storage model
name = db.StringProperty()
date = db.DateTimeProperty(auto_now_add=True)
class mainHandler(webapp2.RequestHandler):
def get(self):
#Champion information acquisition
result = fetch('https://prod.api.pvp.net/api/lol/na/v1.1/champion?api_key='+RIOT_KEY)
if result.status_code == 200:
js = loads(result.content)['champions']
for j in js:
#Champion information storage
modelchampion = modelChampion().get_or_insert(str(j['id']))
modelchampion.name = j['name']
modelchampion.put()
#Copy champion information to Memcache
memcache.add("champion_"+str(j['id']), j['name'], 86399)
app = webapp2.WSGIApplication([ ('/champion', mainHandler) ], debug=True)
Use `memcache.get (key)` to reference the data added to Memcache. Since the data added to Memcache may be lost, it is necessary to describe the processing at that time as well. Now, let's add the champion name to the posted content in tweet.py.
tweet.py
from google.appengine.api import memcache
from champion import modelChampion
def getGame(qid):
q = modelQueue().get_by_id(qid, parent=None)
result = fetch('https://prod.api.pvp.net/api/lol/'+q.resion+'/v1.3/game/by-summoner/'+str(q.summoner_id)+'/recent?api_key='+RIOT_KEY)
if result.status_code == 200:
j = loads(result.content)['games'][0]
if j['stats']['win'] == True:
win = 'victory'
else:
win = 'defeat'
try:
kill = str(j['stats']['championsKilled'])
except:
kill = '0'
try:
death = str(j['stats']['numDeaths'])
except:
death = '0'
try:
assist = str(j['stats']['assists'])
except:
assist = '0'
#Get champion information from Memcache
champion = memcache.get("champion_"+str(j['championId']))
if champion is None: #If you can't get champion information from Memcache, from the data store
champion = modelChampion.get_by_key_name(str(j['championId'])).name
game_type = j['subType']
if j['createDate'] > q.latest_game:
q.latest_game = j['createDate']
q.put()
if tweet(q.summoner_name+'Mr. latest'+game_type+'The battle record is'+champion+'so'+kill+'kill'+death+'death'+assist+'Assist'+win+'soす 。 http://tol.orfx.jp #Tweet_of_Legends', q.access_key, q.access_secret):
q.date_success = datetime.now()
q.put()
This is the first memorandum of app created with GAE. Once you get the hang of it, it's very easy to create an app from the next time, so it's recommended!
Recommended Posts