Note that cibuildwheel builds python bwheel (including C ++ module) in bulk with CI and uploads it to PyPI

If you are a Han, you want to install a prebuilt wheel that includes C ++ modules on Windows, macOS, and Linux with Python pip!

Let's do it with cibuildwheel!

https://github.com/joerick/cibuildwheel

It's a script that builds in various environments and python versions. It's great. You can incorporate this into your own project CI (travis, GitHub Actions, etc).

It's a little difficult to understand,

$ python3 -m cibuildwheel --output-dir wheelhouse

By executing, python setup.py bdist_wheel equivalent will be built and executed for each python version environment. (For Linux and macOS, use Docker).

Run the script before build

In cibuildwheel, Linux and macOS are executed in the Docker container, so when performing CI build with Travis etc., something like git submodule update must be executed inside the docker container.

You can set some in the environment variables of cibuildwheel.

https://cibuildwheel.readthedocs.io/en/latest/options/

Upload to pypi

Notes on uploading packages in Linux binaries to PyPI https://qiita.com/syoyo/items/6185380b8d9950b25561

Please refer to

Azure pipeline ...?

With Azure pipeline, upload automation after build with twine etc. seems to be troublesome at present, but is the setting of vispy helpful?

https://github.com/vispy/vispy/blob/master/azure-pipelines.yml

Templates have been proposed for the time being, but it seems that there are some restrictions.

https://github.com/joerick/cibuildwheel/issues/229

I actually tried using it with tinyobjloader, but the online documentation was hard to understand and it felt more troublesome than Travis. (It would be nice if the yaml description could be syntax checked with an online editor)

If you can use Travis, we recommend using travis (or Github actions).

Build and publish flow

• Whl build with cibuildwheel for each OS
• Use CopyFiles @ 2 to copy the build to the staging folder (disk space left after the job is finished?)
• $ (Build.ArtifactStagingDirectory) is the staging folder
• Name the artifact folder with PublishBuildArtifacsts @ 0

      - task: PublishBuildArtifacts@1
        inputs:
          path: $(Build.ArtifactStagingDirectory)
          artifactName: tinyobjDeployWindows

In deploy (upload to e.g. PyPI), First check that the dependent task is complete (dependsOn) and that it is a tag commit.

  - job: deployPyPI
    # Based on vispy: https://github.com/vispy/vispy/blob/master/azure-pipelines.yml
    pool: {vmImage: 'Ubuntu-16.04'}
    condition: and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/tags/v'))
    dependsOn:
      - linux
      - macos
      - windows

Then use DownloadBuildArtifacts to download the job artifacts (actually folders).

      - task: DownloadBuildArtifacts@0
        inputs:
          artifactName: 'tinyobjDeployWindows'
          downloadPath: $(Pipeline.Workspace)

At this time, a folder is created with ʻartifactName ($ (Pipeline.Workspace) / tinyobjDepoloyWinwdows`), and the build is placed in it. For debugging, it seems good to output it to the log with ls etc. so that you can check it.

Then publish it on twine and you're done!

      # Publish to PyPI through twine
      - bash: |
          cd $(Pipeline.Workspace)
          find .
          python -m pip install --upgrade pip
          pip install twine
          echo tinyobjDeployLinux/python/dist/*
          echo tinyobjDeployLinux/python/wheelhouse/* tinyobjDeployMacOS/python/wheelhouse/* tinyobjDeployWindows/python/wheelhouse/*
          twine upload -u "__token__" --skip-existing tinyobjDeployLinux/python/dist/* tinyobjDeployLinux/python/wheelhouse/* tinyobjDeployMacOS/python/wheelhouse/* tinyobjDeployWindows/python/wheelhouse/*
        env:
          TWINE_PASSWORD: $(pypiToken2)

For the TWINE password (secret token), register the secret token with variable on the Azure Pipeline website with the name pypiToken2.

Impressions

I do git push every time, trigger the build, wait for about 10 minutes, fix it if an error occurs & attach the tag again and push is troublesome, so I want an environment where I can try it at hand.

Twine token

Azure Pipeline also has a secret variables function that can handle Twine (PyPi) secret tokens, so you can use that.

https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#secret-variables

The Secret variable is specified via an environment variable in the build script.

Use PublishBuildArtifacts

PublishPipelineArtifact only prints logs that aren't even the error message ʻOne or more errors occurred` (no error reason is displayed) and it doesn't work.

https://stackoverflow.com/questions/58841733/how-to-debug-azure-devops-task-publishpipelineartifact-when-one-or-more-errors-o

Use PublishBuildArtifacts.

Actual sample with C ++ code

cibuildwheel, pypi upload,

pytinyexr (PyEXR) uses Travis.

https://github.com/syoyo/PyEXR

tinyobjloader uses Azure Pipeline.

https://github.com/tinyobjloader/tinyobjloader/blob/master/azure-pipelines.yml

Py2.7(or pypy) + Windows + pybind11

At least pypy will cause a build error, so be sure to include pp27-win32 and pp36-win32 in CIBW_SKIP and do not build. (Is anyone using pypy?)

Also, python2.7 + C ++ 11 is basically deprecated.

https://cibuildwheel.readthedocs.io/en/stable/cpp_standards/

Unless you have a strong reason to use py27, you shouldn't support python 2.7.