[PYTHON] Prepare a pseudo API server using GitHub Actions

Introduction

Maybe a little more recently, a feature called GitHub Actions has been added. This is the one that can fire the specified Actions (execution of shell script) for the specified event. For example, if the PR to the dev branch is merged, build it and push it to the dev-pages branch, which is a function for so-called CI / CD.

This time, I participated in the Hokkaido version of the COVID-19 countermeasure site, which is currently a big move, and developed using this function. Taking this as an example, is it possible to make GitHub a pseudo API server as the title suggests? I will make a proposal in this article.

Scheduling with GitHub Actions

How it came to Actions

https://github.com/Kanahiro/covid19hokkaido_scraping This is a script created to support the JSON format data that is the source of visualization when developing the above-mentioned countermeasure site. When executed (in the version at that time), it has the function of patrolling the website of the road, reading the static CSV file held in the branch and outputting the json file (currently reading the external CSV file). ). Initially, for the time being, I had to manually run main.py with the script I wrote. At that time, a member of JUST Road IT suggested "Can you do scheduling?" At this point, I didn't know if I could do it, so I didn't reply for the time being, but when I secretly investigated the implementation method, it was surprisingly easy, so I implemented it (Actions is God).

Scheduling method

Reference site: Use GitHub Actions as a scheduler

Actions are defined in the yaml file. Defines the timing "on" to fire first. For normal CI / CD use, it can be triggered by pushing to a specific branch. However, by writing as follows, it can be executed every 15 minutes.

on:
  schedule:
    - cron:  '*/15 * * * *'

#(It's a secret that cron? What's that?)

Scheduling is now complete. Then, run the above script every 15 minutes and push the json data generated in another branch! So, I decided to schedule with the following yaml file.

name: Python application

on:
  schedule:
    - cron:  '*/15 * * * *'

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: Set up Python 3.8
      uses: actions/setup-python@v1
      with:
        python-version: 3.8
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Run script
      run: |
        python main.py #Run the main script
    - name: deploy
      uses: peaceiris/actions-gh-pages@v3
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        publish_dir: ./data
        publish_branch: gh-pages

I wrote it with reference to another person's yaml file, so I cried when I was told to write it in full scratch. I thought it was amazing that you could install dependencies with pip. When main.py is executed, it outputs json files in the data directory in the same directory. After processing the script, push the files in the data directory to gh-pages (authorized by secrets.GITHUB_TOKEN) So, gh-pages now contains json files that are generated once every 15 minutes.

As a pseudo API server

I think it's a title scam, but the pseudo API server I propose is complete. https://raw.githubusercontent.com/Kanahiro/covid19hokkaido_scraping/gh-pages/patients.json This is a json file that Actions automatically generates every 15 minutes on the above script repository. This file doesn't seem to be subject to CORS limits (although I haven't verified it myself yet), so it can be read directly at the front desk. In other words, if you get this file and "color" the elements on the front side, you can actually treat it as an API server!

It was a proposal. I think that it is effective when there is no resource to build a separate API server, it is redundant, or it is troublesome. Also, I think it's good to have a better outlook to complete everything on GitHub without interposing other services.

Postscript: About GitHub Pages

The destination of the output json mentioned above was the gh-pages branch, but for some reason I could not access it via github.io, so I was directly accessing the above raw.githubusercontent.com. However, in the comments, the following exchanges were made.

Hmm…? I feel like an account name that I'm familiar with ...

The Actions script from earlier

      uses: peaceiris/actions-gh-pages@v3
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        publish_dir: ./data
        publish_branch: gh-pages

(This person (@peaceiris) is the person who made the action to push to gh-pages used in the script ...) So, I chose to output to gh-pages again from settings ...

https://kanahiro.github.io/covid19hokkaido_scraping/patients.json

I was able to host on gh-pages safely, I'm sorry (and thank you). So, after setting Actions, you need to check settings to host gh-pages again, so please be careful.