Build a test flow on CircleCI using Jib

TL;DR

--A brief introduction of Jib published by Google - https://github.com/GoogleContainerTools/jib --Use Jib and CircleCI to build a flow to push a black-box tested Docker image to ECR (Amazon EC2 Container Registry).

What is Jib

In order to make a Java application into a Docker container with Docker, you need to take steps such as building Java, preparing a Dockerfile, and installing Docker. Jib is an OSS that can perform all such troublesome processes with a single command. Published as a Maven and Gradle plugin.

Try using Jib as per the manual.

Added the following description to build.gradle

plugins {
  id 'com.google.cloud.tools.jib' version '0.10.1'
}

After writing, execute the command as follows according to the manual

gradle jib --image=Image name you want to generate

By doing just this, a Docker image will be generated and pushed to Dockerhub. You don't even need a Docker daemon, so you don't need Docker in your environment. ** Works with Gradle. However, if you run it in an environment without Docker, you can authenticate to Dockerhub with just the above description. To run without Docker locally, you need to set authentication in bug.gradle. If you have Docker locally, you can push it without writing the setting in build.gradle if you execute docker login in advance and authenticate.

You may also want to create a Docker image locally. If that happens

gradle jibDockerBuild --image=Image name you want to generate

It is also possible to generate it on local Docker by doing.

What I wanted to do

Jib even pushes the image to the Docker repository, making it easier than manually generating the container image from Docker. Taking advantage of being able to easily containerize without preparing a Dockerfile etc., I will try to incorporate it into the CI flow. We will perform a black box test on the Docker image generated by Jib, and if there is a problem, push it to ECR, and if there is a problem, we will build a mechanism to raise an alert without sending it to ECR.

Automation flow

  1. Change the code and push to GitHub. CircleCI runs
  2. Run a white box test on CircleCI
  3. Run Jib and generate a Docker image on Docker on CircleCI
  4. Launch a container from the generated image and run a black box test
  5. If all of the above are okay, use Jib to push to ECR. Notify Slack if there is a problem

1. Change the code and push to GitHub. CircleCI runs

First, let's integrate your GitHub repository with CircleCI. Integrate GitHub and CircleCI Ready if you cooperate with reference to

2. Run white box test

This is a basic part, so I will omit it.

3. Run Jib and generate a Docker image on Docker on CircleCI

Put the following description in build.gradle and execute the command to create an image in local Docker.

build.gradle


jib {
  from {
    //Base image
    image = 'circleci/java'
  }
  to {
    image = 'ECR repository to push to'
  }
}
gradle jibDockerBuild

Since the to setting of jib is added on gradle, --image is unnecessary.

4. Launch a container from the generated image and run a black box test

Launch a container from a Docker image and do a black box test, but there are some caveats. Since Docker on CircleCI is not on CircleCI's host, even if you try to access it from the launched host, you cannot.

Take the case of launching an Nginx container on CircleCI as an example.

Example in Nginx


docker run -d -p 8080:80 nginx
curl localhost:8080
curl: (7) Failed to connect to localhost port 8080: Connection refused

Therefore, prepare a Docker image to be tested against the Docker container and test it. In short, it is just an image of sending a request to a specific IP (this time the IP of the container to be tested) and testing it. Let's make this quickly with jib and execute it.

yml:.circle/config.yml


      - run:
          name: create test image
          command: cd ./test ; ../gradlew jibDockerBuild
      - run:
          name: blackbox test
          command:Image to run docker run test

5. 5. If all of the above are correct, use Jib to push to ECR. Notify Slack if there is a problem

When pushing from Jib to ECR, use amazon-ecr-credential-helper. Therefore, you must add amazon-ecr-credential-helper to your PATH before running.

yml:.circle/config.yml


      - run:
          name: download go
          command: wget -O - "https://redirector.gvt1.com/edgedl/go/go1.9.2.linux-amd64.tar.gz" | tar zxvf - -C /home/circleci/project
      - run:
          name: get ecr-helper
          command: ./go/bin/go get -u github.com/awslabs/amazon-ecr-credential-helper/ecr-login/cli/docker-credential-ecr-login
      - run:
          name: push ecr
          command: export PATH=$PATH:/home/circleci/go/bin/ ; ./gradlew jib

The above description is not efficient because it is dropped for each Go language every time.

If you want to pass the certification to GCP and Dockerhub, please check the Official Document.

Notification to Slack is set by referring to Notify Slack of CircleCI results.

Sample code

https://github.com/supecura/JibSample

Which area is better

--No need to manage Docker-related files such as Dockerfile ――However, if you prepare your own image to use as a base, you need to manage it. --There is also a way to leave everything to CircleCI as in the above example. The build time will be extended by the amount you leave it to me, so it has advantages and disadvantages. --Problem discovery is quicker ――I have the impression that the black box test is done before the production environment is prepared and BlueGreen is deployed, but the above flow is faster to find the problem. ――Also, even for services that are deployed in large numbers, if you find them after preparing the production environment, it will be quite troublesome to rewind, but it will also help prevent such situations.

bonus

Questions that came up when I actually tried to introduce it and the presence or absence of its function

--Can Java execution classes and arguments be set? --Can I change the JVM arguments? ――Can you set environment variables?

These can be set from Jib's container property.

sample


jib {
  container {
    jvmFlags = ['-Xms512m']
    mainClass = 'Execution class'
    args = ['hoge','huga']
    environment = [ENV:'develop', LANG:'ja_JP.UTF-8']
  }
}

For other detailed settings, go to Official Documents

Recommended Posts

Build a test flow on CircleCI using Jib
multi-project docker build using jib
Build a Minecraft server on AWS
Build a XAMPP environment on Ubuntu
Using a local network on iOS 14
Build a browser test environment using Capybara in the Docker development environment
How to build a Ruby on Rails environment using Docker (for Docker beginners)
Build a Maven repository on AWS S3
Build a Java development environment on Mac
Build a JMeter environment on your Mac
[When using MiniMagick] A memorandum because I stumbled in the CircleCI test environment.
Build a Kotlin app using OpenJDK's Docker container
Generate a test account password using Rails SecureRandom.alphanumeric
Build a DHCP and NAT router on Ubuntu 16.04
Build a streaming server on your iOS app
Introducing Rspec, a Ruby on Rails test framework
Build a Laravel environment on an AWS instance
Build a Java runtime environment on Sakura VPS
How to build a Pytorch environment on Ubuntu
Install Ubuntu 20.04 in virtual box on windows10 and build a development environment using docker
Create a Java development environment using jenv on Mac
Memo to build a Servlet environment on AWS EC2
Build a Ruby on Rails development environment on AWS Cloud9
Easily build Redmine on Windows using WSL2 and Docker
[JUnit 5 compatible] Write a test using JUnit 5 with Spring boot 2.2, 2.3
I tried using Log4j2 on a Java EE server
Build a Maven in-house repository on Google Cloud Storage
[Docker] Build an Apache container on EC2 using dockerfile
A record of a brief touch on GCP's Data Flow
Try to build a Java development environment using Docker
[2021] Build a Docker + Vagrant environment for using React / TypeScript