Build a test flow on CircleCI using Jib

TL;DR

--A brief introduction of Jib published by Google - https://github.com/GoogleContainerTools/jib --Use Jib and CircleCI to build a flow to push a black-box tested Docker image to ECR (Amazon EC2 Container Registry).

What is Jib

In order to make a Java application into a Docker container with Docker, you need to take steps such as building Java, preparing a Dockerfile, and installing Docker. Jib is an OSS that can perform all such troublesome processes with a single command. Published as a Maven and Gradle plugin.

• This time, we will use the Gradle plugin. Not only that, Jib runs faster than building a container from Docker because it loads Docker's build settings and manipulates only the layers that have changed.

Try using Jib as per the manual.

Added the following description to build.gradle

plugins {
  id 'com.google.cloud.tools.jib' version '0.10.1'
}

After writing, execute the command as follows according to the manual

• Gradle version 4.6 or later is required.

gradle jib --image=Image name you want to generate

By doing just this, a Docker image will be generated and pushed to Dockerhub. You don't even need a Docker daemon, so you don't need Docker in your environment. ** Works with Gradle. However, if you run it in an environment without Docker, you can authenticate to Dockerhub with just the above description. To run without Docker locally, you need to set authentication in bug.gradle. If you have Docker locally, you can push it without writing the setting in build.gradle if you execute docker login in advance and authenticate.

You may also want to create a Docker image locally. If that happens

gradle jibDockerBuild --image=Image name you want to generate

It is also possible to generate it on local Docker by doing.

What I wanted to do

Jib even pushes the image to the Docker repository, making it easier than manually generating the container image from Docker. Taking advantage of being able to easily containerize without preparing a Dockerfile etc., I will try to incorporate it into the CI flow. We will perform a black box test on the Docker image generated by Jib, and if there is a problem, push it to ECR, and if there is a problem, we will build a mechanism to raise an alert without sending it to ECR.

Automation flow

1. Change the code and push to GitHub. CircleCI runs
2. Run a white box test on CircleCI
3. Run Jib and generate a Docker image on Docker on CircleCI
4. Launch a container from the generated image and run a black box test
5. If all of the above are okay, use Jib to push to ECR. Notify Slack if there is a problem

1. Change the code and push to GitHub. CircleCI runs

First, let's integrate your GitHub repository with CircleCI. Integrate GitHub and CircleCI Ready if you cooperate with reference to

• Preparation Create a user who has permission from AMI to ECR, and register the access key of the created user in Environment Variables to AWS_ACCESS_KEY_ID and the secret key to AWS_SECRET_ACCESS_KEY.

2. Run white box test

This is a basic part, so I will omit it.

3. Run Jib and generate a Docker image on Docker on CircleCI

Put the following description in build.gradle and execute the command to create an image in local Docker.

build.gradle

jib {
  from {
    //Base image
    image = 'circleci/java'
  }
  to {
    image = 'ECR repository to push to'
  }
}

gradle jibDockerBuild

Since the to setting of jib is added on gradle, --image is unnecessary.

4. Launch a container from the generated image and run a black box test

Launch a container from a Docker image and do a black box test, but there are some caveats. Since Docker on CircleCI is not on CircleCI's host, even if you try to access it from the launched host, you cannot.

Take the case of launching an Nginx container on CircleCI as an example.

Example in Nginx

docker run -d -p 8080:80 nginx
curl localhost:8080
curl: (7) Failed to connect to localhost port 8080: Connection refused

Therefore, prepare a Docker image to be tested against the Docker container and test it. In short, it is just an image of sending a request to a specific IP (this time the IP of the container to be tested) and testing it. Let's make this quickly with jib and execute it.

yml:.circle/config.yml

      - run:
          name: create test image
          command: cd ./test ; ../gradlew jibDockerBuild
      - run:
          name: blackbox test
          command:Image to run docker run test

5. 5. If all of the above are correct, use Jib to push to ECR. Notify Slack if there is a problem

When pushing from Jib to ECR, use amazon-ecr-credential-helper. Therefore, you must add amazon-ecr-credential-helper to your PATH before running.

yml:.circle/config.yml

      - run:
          name: download go
          command: wget -O - "https://redirector.gvt1.com/edgedl/go/go1.9.2.linux-amd64.tar.gz" | tar zxvf - -C /home/circleci/project
      - run:
          name: get ecr-helper
          command: ./go/bin/go get -u github.com/awslabs/amazon-ecr-credential-helper/ecr-login/cli/docker-credential-ecr-login
      - run:
          name: push ecr
          command: export PATH=$PATH:/home/circleci/go/bin/ ; ./gradlew jib

The above description is not efficient because it is dropped for each Go language every time.

• This time, we are realizing it without using Docker related files. If you prepare your own CircleCI base image, it will be executed smoothly without DL every time.

If you want to pass the certification to GCP and Dockerhub, please check the Official Document.

Notification to Slack is set by referring to Notify Slack of CircleCI results.

Sample code

https://github.com/supecura/JibSample

Which area is better

--No need to manage Docker-related files such as Dockerfile ――However, if you prepare your own image to use as a base, you need to manage it. --There is also a way to leave everything to CircleCI as in the above example. The build time will be extended by the amount you leave it to me, so it has advantages and disadvantages. --Problem discovery is quicker ――I have the impression that the black box test is done before the production environment is prepared and BlueGreen is deployed, but the above flow is faster to find the problem. ――Also, even for services that are deployed in large numbers, if you find them after preparing the production environment, it will be quite troublesome to rewind, but it will also help prevent such situations.

bonus

Questions that came up when I actually tried to introduce it and the presence or absence of its function

--Can Java execution classes and arguments be set? --Can I change the JVM arguments? ――Can you set environment variables?

These can be set from Jib's container property.

sample

jib {
  container {
    jvmFlags = ['-Xms512m']
    mainClass = 'Execution class'
    args = ['hoge','huga']
    environment = [ENV:'develop', LANG:'ja_JP.UTF-8']
  }
}

For other detailed settings, go to Official Documents