[PYTHON] I tried to notify the update of "Become a novelist" using "IFTTT" and "Become a novelist API"

Introduction

I've been indebted to becoming a novelist, but I often waste time waiting for updates in front of the screen ... Even if I knew it was useless, I expected it to reload, so I decided to make an app that would check for updates. (Addition) Automation was achieved with Task Scheduler. It works only when the computer is turned on, but it's convenient! (August 9, 2020)

What do you do?

Get information on the latest story of the novel using the Naro API. After that, LINE Notify will send a notification if there is any updated information by comparing it with the previously acquired information locally. narou.PNG

environment

Before preparing ... Check and note about the difference between API and scraping

This time, we will use the official API, so we have not scraped it. </ B> But if you're worried, check out the law.

Preparation (Create Applet with IFTTT )

It is a service that links services different from IFTTT. This time, connect Webhooks and LINE Notify and have them send notifications to your LINE. Procedure </ b>

  1. Register with IFTTT
  2. Click Create at the top right of the screen. The screen will change to the screen that says "if + This Then That". See the figure below. ifttt_intro.PNG
  3. Click + This. Type Webhooks into the search bar to select it.
  4. Click on the column that says "Receive a web request". When you come to the screen below, enter your favorite name in "Event Name". I will use it later. ifttt_2.PNG
  5. Click + That. Select LINE and click the "Send message" field.
  6. Log in to LINE, set the content of the message to "Value1: Value1 \
    " (it doesn't matter if you don't) and click "Create Action". ifttt_3.PNG
  7. Check the contents and click Finish.
  8. Click Explore, type Webhooks in the search window and select the Services tab. Click Webhooks. Maybe you can go from this link ...?
  9. Click Documentation and you should see "Your key is: ~~~~~", so make a note of it. I will use it later. This is the end of IFTTT.

Source code description

I posted it on GitHub ( here ). A brief description of each function is given. import

import requests
import yaml
import gzip
import json
import csv

get_new_data() Narurou API is used to specify the novel ['ncode','title','general_lastup','general_all_no '] Is acquired. Please refer to the official website and the article below for how to use the Naro API. I think it's interesting because it has various uses.

def get_new_data():
    """
    payload : json params
    e.g.) payload = {'out': 'json', 'ncode':Please enter the ncode of the novel you are reading, 'of':'t-gl', 'gzip':5}
    reference) https://dev.syosetu.com/man/api/
    """
    payload = {'out': 'json', 'ncode': """Please enter ncode""", 'of':'t-gl-n-ga', 'gzip':5}
    url = "https://api.syosetu.com/novelapi/api/"
    r_gzip = requests.get(url, params=payload)
    r = r_gzip.content
    r_gzip.encoding = 'gzip'
    res_content = gzip.decompress(r).decode("utf-8")
    res = json.loads(res_content)[1:]
    
    new_data = [['ncode', 'title', 'general_lastup', 'general_all_no']]

    for r in res:
        new_data.append([r['ncode'], r['title'], r['general_lastup'], str(r['general_all_no'])])

    return new_data

read_data() Read the information such as the total number of stories and the update date of the novel obtained last time from the file update.csv. When you run this code for the first time, update.csv should be blank, so if you try to use it as it is, an error will naturally occur. Before that, create update.csv using get_new_data () and store_data () described later.

def read_data():
    f_data = "update.csv"
    with open(f_data, encoding="utf-8") as f:
        reader = csv.reader(f)
        data = [row for row in reader]
    #data: list(['ncode', 'title', 'general_lastup', 'general_all_no'])
    return data

store_data(res) Write the novel information stored in the list res to update.csv.

def store_data(res):
    # res: list(['ncode', 'title', 'general_lastup', 'general_all_no'])
    output = "update.csv"
    with open(output, mode='w', newline="", encoding="utf-8") as f:
        writer = csv.writer(f)
        for d in res:
            writer.writerow([d[0], d[1], d[2], d[3]])

post_ifttt(json) Use the Applet name and Webhooks key you noted earlier here. Substitute it in the commented out location. When this function is called, LINE will be notified.

def post_ifttt(json):
    # json: {value1: " content "}
    url = (
        "https://maker.ifttt.com/trigger/"
        + #Applet name
        + "/with/key/"
        + #Webhooks Key
    )
    requests.post(url, json)

check(prev_data, new_data) Check if it has been updated. For updated novels, call post_ifttt () to notify you.

def check(prev_data, new_data):
    # prev_data: [[previous information of bookmark1], [previous information of bookmark2], ...]
    # new_data: [[new information of bookmark1], [new information of bookmark2], ...]
    isUpdated = 0
    for i in range(len(prev_data)):
        if(i==0):
            continue
        if(prev_data[i][0]==new_data[i][0] and prev_data[i][2]!=new_data[i][2]):
            message = '\"' + new_data[i][1] + '\"' +  'Has been updated!\n' + 'https://ncode.syosetu.com/' + new_data[i][0] + '/' + new_data[i][3] + '/'
            print(message)
            json = {'value1': message)
            post_ifttt(json)
    store_data(new_data)

Variables for storing data at runtime </ b>

prev_data = read_data()
new_data = get_new_data()
check(prev_data, new_data)

Impressions made

I feel that API is convenient. Before writing the program, I was wondering, "Do I have to do scraping? ... The story around the law seems to be confusing, and I'm afraid, how do I do scraping first?" It was completed by implementing only a few. I wasn't happy. You can make a simple app just by doing what you learned in school lessons. Of course, I think there is room for improvement. If you scrape from My Page without using the API, you can get information on your favorite novels without ncoding. Even if you use the API, it seems to be a learning experience to acquire the contents of the novel together and make it for reading as a smartphone application (although some people have already done it).

References

Recommended Posts