background purpose

I've summarized how to bring an internal library from another private repository when building a Python image on Github Actions.

I already had an article by my colleague @ elyunim26, "Securely browse GitHub private repository in a container built with GitHub Actions", but I'm not a machine user, but a [deploy key]( Since I was using https://docs.github.com/ja/free-pro-team@latest/developers/overview/managing-deploy-keys), I needed a secure method for ssh authentication. You can also safely use the ssh key to build an image by using Build-time secrets.

Implementation method

Briefly, it can be achieved by the following procedure.

Read the deploy key (private key) from secrets with Github Actions
Securely pass using Docker's Buildkit function
Do pip install as usual

Deploy key settings

First, you need to set the deploy key in the repository on the library side. Please refer to Official Github documentation for how to set the deploy key.

Github Actions settings

You need to be aware of the following factors: Installing a node module in a private repository with GitHub Actions was helpful for troubleshooting (although there is a difference between Python and node.js).

Secrets starting with GITHUB are prohibited
Since line breaks cannot be used in secrets, it is necessary to register in advance by replacing line breaks with \ n etc. and replace them with sed.

The following is a setting example. Since our team uses AWS, we also include the code to push to ECR later.

`github_actions_ecr.yml`


      - name: Build, tag, and push image to Amazon ECR
        id: build-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: ${{ steps.extract_repository.outputs.repository }}
          IMAGE_TAG: ${{ github.sha }}
          DEPLOY_KEY_GITHUB: ${{ secrets.DEPLOY_KEY_GITHUB }}
        run: |
          echo ${DEPLOY_KEY_GITHUB} > .deploy_key
          sed -i -e "s#\\\\n#\n#g" .deploy_key
          chmod 600 .deploy_key
          DOCKER_BUILDKIT=1 docker build --secret id=ssh,src=.deploy_key \
            -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG \
            -t $ECR_REGISTRY/$ECR_REPOSITORY:latest \
            .
          rm .deploy_key
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest

Dockerfile settings

Our team often uses poetry, so we use pyproject.toml, but if you use requirements.txt, you should be able to use the same settings.

`Dockerfile`


# syntax=docker/dockerfile:experimental
FROM python:3.7

# ssh-Due to keyscan execution, openssh depending on the original image-client needs to be installed
RUN apt-get install -y openssh-client git

COPY ./pyproject.toml /app/pyproject.toml
COPY ./src /app/src
WORKDIR /app

RUN mkdir -m 700 $HOME/.ssh
RUN ssh-keyscan -H github.com > $HOME/.ssh/known_hosts
RUN --mount=type=secret,id=ssh,dst=$HOME/.ssh/id_rsa \
  pip install --upgrade pip && \
  pip install . && rm pyproject.toml

In pyproject.toml, it is described as follows. This is created by specifying the repository with the poetry add command, as described in the official poetry documentation.

`pyproject.toml`


[tool.poetry.dependencies]
{Library name} = {git = "ssh://[email protected]/{username}/{Library name}.git", rev = "main"}

Build a Docker image containing the private repository Python library on GitHub Actions