[PYTHON] I built a Wheel for Windows using Github Actions

I maintain a client library called mysqlclient that uses libmysqlclient for Python.

This library provides a statically linked Binary Wheel for libmysqlclient for easy use by Windows users. I tried to automate the build of this Binary Wheel using Github Actions.

I'm new to Github Actions, so there may be a better way, but I hope it helps someone.

Impressions

I will write from the impression first. I've already used AppVeyor to auto-build Wheels for Windows in other projects, and it's far more comfortable than that.

• VM startup is fast --I felt that the time from pushing to VM startup was overwhelmingly shorter than AppVeyor. When I was using AppVeyor, I had to wait a few minutes for each push, but with Github Actions I was able to do a lot of trial and error.
• Cache Hurray --This time there was a dependent library that took a few minutes to build, but by caching it, I was able to proceed with trial and error of the main body quickly.
• You can choose a different shell for each step --cmd, PowerShell, and Git for Windows bash can be selected and written for each step. I'm new to cmd and PowerShell, so it was easy to get away with bash right away when I was wondering what to do.
• Artifact is easy --You can register and download the built Wheel as Artifact. This could be done with AppVeyor, but Github Actions is easier to download, and it's zipped so you don't have to download multiple files one by one.

(Some of these may actually be the same with AppVeyor. It's just a personal impression.)

Prepare a repository for build

Instead of using the repository of the library itself, I created another repository mysqlclient-build for build and tried and errored there. This allows you to do trial and error without polluting the library repository.

.github/workflows/windows.yaml

name: Build windows wheels
on:
  push:
    branches:
      - master
  create:

As I will explain later, this affected the directory to be checked out and it was a little troublesome, so I feel that it was okay to make a trial and error with a pull request of the repository of the library itself.

Build dependent libraries

mysqlclient uses libmysqlclient, but Wheel for Windows uses a compatible MariaDB Connector / C instead of MySQL Connector / C.

This is because it can be built to use the Windows API instead of OpenSSL for SSL connections to MySQL, sha256 calculations for caching_sha2_password, etc., which is suitable for statically linked binary distribution.

However, MariaDB Connector / C, which is distributed in binary, has plugins such as authentication as DLLs. I want to make it a single binary if possible, so I build MariaDB Connector / C myself.

Use the cache action because it takes time to build this dependent library every time you build mysqlclient. The cache action caches the specified directory when the job succeeds, so push it once until the MariaDB Connector / C build, make it successful, and cache it. Now I was able to take advantage of the pre-built MariaDB Connector / C when doing the trial and error to build mysqlclient.

.github/workflows/windows.yaml

jobs:
  build:
    runs-on: windows-latest
    env:
      CONNECTOR_VERSION: "3.1.5"

    steps:
      - name: Cache Connector
        id: cache-connector
        uses: actions/cache@v1
        with:
          path: c:/mariadb-connector
          key: mariadb-connector-3.1.5-win

      - name: Download and Unzip Connector
        if: steps.cache-connector.outputs.cache-hit != 'true'
        shell: bash
        run: |
          curl -LO "https://downloads.mariadb.com/Connectors/c/connector-c-${CONNECTOR_VERSION}/mariadb-connector-c-${CONNECTOR_VERSION}-src.zip"
          unzip "mariadb-connector-c-${CONNECTOR_VERSION}-src.zip" -d c:/
          mv "c:/mariadb-connector-c-${CONNECTOR_VERSION}-src" c:/mariadb-connector-src

      - name: Build Connector
        if: steps.cache-connector.outputs.cache-hit != 'true'
        shell: cmd
        working-directory: c:/mariadb-connector-src
        run: |
          mkdir build
          cd build
          cmake -A x64 .. -DCMAKE_BUILD_TYPE=Release -DCLIENT_PLUGIN_DIALOG=static -DCLIENT_PLUGIN_SHA256_PASSWORD=static -DCLIENT_PLUGIN_CACHING_SHA2_PASSWORD=static
          cmake --build . -j 8 --config Release
          cmake -DCMAKE_INSTALL_PREFIX=c:/mariadb-connector -DCMAKE_INSTALL_COMPONENT=Development -DCMAKE_BUILD_TYPE=Release -P cmake_install.cmake

Build mysqlclient

First, check out mysqlclient with the checkout action. By default, the repository with workflow (mysqlclient-build in this case) is checked out to the working directory, but you can use with: to check out another repository to another path.

There was one pitfall here, but if you check out with path: mysqlclient, the checkout destination will be ../ mysqlclient instead of ./mysqlclient. This is because the default working directory is the directory for checking out that repository, and when checking out another repository, the directory is created next to it instead of under it and checked out. So, in the steps after checking out, working-directory: ../mysqlclient is specified.

After building, use the action upload-artifact to upload the wheel to Github, and then make sure you can actually install and import the wheel.

I wanted to test it to work if possible, but I haven't done it this time because windows-latest doesn't have MySQL Server installed. It seems that Docker can be used, so I will try to set up MySQL Server using Docker in the future.

.github/workflow/windows.yml

      - name: Checkout mysqlclient
        uses: actions/checkout@v1
        with:
          repository: PyMySQL/mysqlclient-python
          ref: master
          fetch-depth: 10
          path: mysqlclient

      - name: Site Config
        shell: bash
        working-directory: ../mysqlclient
        run: |
          pwd
          find .
          cat <<EOF >site.cfg
          [options]
          static = True
          connector = C:/mariadb-connector
          EOF
          cat site.cfg

      - name: Build wheels
        shell: cmd
        working-directory: ../mysqlclient
        run: |
          py -3.8 -m pip install -U setuptools wheel pip
          py -3.8 setup.py bdist_wheel
          py -3.7 -m pip install -U setuptools wheel pip
          py -3.7 setup.py bdist_wheel
          py -3.6 -m pip install -U setuptools wheel pip
          py -3.6 setup.py bdist_wheel

      - name: Upload Wheel
        uses: actions/upload-artifact@v1
        with:
          name: win-wheels
          path: ../mysqlclient/dist

      - name: Check wheels
        shell: bash
        working-directory: ../mysqlclient/dist
        run: |
          ls -la
          py -3.8 -m pip install mysqlclient-1.4.6-cp38-cp38-win_amd64.whl
          py -3.8 -c "import MySQLdb"
          py -3.7 -m pip install mysqlclient-1.4.6-cp37-cp37m-win_amd64.whl
          py -3.7 -c "import MySQLdb"
          py -3.6 -m pip install mysqlclient-1.4.6-cp36-cp36m-win_amd64.whl
          py -3.6 -c "import MySQLdb"