Serverless Java EE starting with Quarkus and Cloud Run

Introduction

Cloud Run was released on Google Cloud Next 19 the other day. This is Google's fully managed GCP offering Knative that autoscales Docker containers launched on HTTP on k8s. AWS Fargate, Heroku, or we Did you say the GAE Flexible Environment that you really wanted?

This time, I will make a high-speed Serverless Java EE application by using Quarkus, a Java EE container that sings supersonic / ultra-lightweight that can compile Java into binary with Graal. .. Don't let me say ** "Spin-up is slow because it's Java" **!

TL;DR

-Clone this product --Build Dockerfile.gcp and deploy to Cloud Run --Quarkus fast! Cloud Run Easy!

Develop sample application with Quarkus

What is Quarkus?

Quarkus is a next-generation Java EE container created by Redhat, and it has a feature that it can be called "Supersonic Subatomic Java" and it starts at a different dimension speed of ** 10ms **, which is unrivaled by others. I will. The main features are as follows.

--Since it supports MicroProfile, you can use basic Java EE functions such as JAX-RS and CDI / JPA. --Since it is executed as a Linux binary instead of a JVM using native-image of GraalVM, the startup speed is equivalent to that of Go language. --One configuration, Hot Reload, OpenAPI, Docker / Maven / Gradle support, etc. Easy to develop as a simple Java EE environment

The details have become longer, so I have summarized them in a separate article. If you are interested, please read this as well. "Blog What is it: JavaEE container in the Serverless era-Quarkus"

Sample app specifications

Now, let's create a sample application. Anything is fine, but this time I will try to make "** Bank API **".

The functions are as follows.

--You can create an account --You can check the list of accounts --You can deposit money in your account --You can withdraw from your account

In addition, since user management is not performed for simplification, anyone can deposit and withdraw w We will also store the account information in the database.

Creating a project

Projects can be created with maven or gradle. This time I will use maven.

% mvn io.quarkus:quarkus-maven-plugin:create
...
Set the project groupId [org.acme.quarkus.sample]: cn.orz.pascal.mybank
Set the project artifactId [my-quarkus-project]: my-bank
Set the project version [1.0-SNAPSHOT]: 
Do you want to create a REST resource? (y/n) [no]: y
Set the resource classname [cn.orz.pascal.mybank.HelloResource]: 
Set the resource path  [/hello]: 
...
[INFO] Finished at: 2019-04-14T17:51:48-07:00
[INFO] ------------------------------------------------------------------------

Added / hello as a sample endpoint. The code for JAX-RS is as follows.

@Path("/hello")
public class HelloResource {
    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() {
        return "hello";
    }
}

It's a normal JAX-RS code, isn't it? Let's run it. Start in development mode.

$ mvn compile quarkus:dev
...
2019-04-14 17:52:13,685 INFO  [io.quarkus](main) Quarkus 0.13.1 started in 2.042s. Listening on: http://[::]:8080
2019-04-14 17:52:13,686 INFO  [io.quarkus](main) Installed features: [cdi, resteasy]

I will try to access it with curl.

$ curl http://localhost:8080/hello
hello 

You have successfully confirmed the access.

DB environment preparation

Next, create an Account table that represents your account. First of all, it is necessary to prepare the DB, so start postgres with Docker.

$ docker run -it -p 5432:5432 -e POSTGRES_PASSWORD=mysecretpassword postgres        
...
2019-04-15 01:29:51.370 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2019-04-15 01:29:51.370 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2019-04-15 01:29:51.374 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2019-04-15 01:29:51.394 UTC [50] LOG:  database system was shut down at 2019-04-15 01:29:51 UTC
2019-04-15 01:29:51.404 UTC [1] LOG:  database system is ready to accept connections

JPA settings

Next, set JPA. First, add dependencies.

#Confirmation of Extension name
$ mvn quarkus:list-extensions|grep jdbc         
[INFO] 	 * JDBC Driver - H2 (io.quarkus:quarkus-jdbc-h2)
[INFO] 	 * JDBC Driver - MariaDB (io.quarkus:quarkus-jdbc-mariadb)
[INFO] 	 * JDBC Driver - PostgreSQL (io.quarkus:quarkus-jdbc-postgresql)
$ mvn quarkus:list-extensions|grep hibernate
[INFO] 	 * Hibernate ORM (io.quarkus:quarkus-hibernate-orm)
[INFO] 	 * Hibernate ORM with Panache (io.quarkus:quarkus-hibernate-orm-panache)
[INFO] 	 * Hibernate Validator (io.quarkus:quarkus-hibernate-validator)
#Add Extension
$ mvn quarkus:add-extension -Dextensions="io.quarkus:quarkus-jdbc-postgresql"
$ mvn quarkus:add-extension -Dextensions="io.quarkus:quarkus-hibernate-orm"

Next, describe the DB settings in src / main / resources / application.properties. Basically, Quarkus recommends to list everything in this ʻapplication.properties` without defining another configuration file such as persistance.xml or log4j.xml. Also, since microprofile-config is used for this file, it can be overwritten with environment variables and arguments. In other words, the difference between the development environment and STG or Prod can be defined on the yaml side of k8s, so there is no need for build methods such as switching settings in Profile for each environment, and operation can be simplified.

This is a great match for environments that meet Twelve-Factor like Dokcer, Cloud Run or Heroku, which specify environment variables at startup / deployment. ..

Click here for the settings to be added to ʻapplication.properties`. As you can see, it is the DB setting.

# datasource account
quarkus.datasource.url: jdbc:postgresql://localhost:5432/postgres
quarkus.datasource.driver: org.postgresql.Driver
quarkus.datasource.username: postgres
quarkus.datasource.password: mysecretpassword

# database optional
quarkus.hibernate-orm.database.generation=drop-and-create
quarkus.hibernate-orm.log.sql: true

Add Entity / Service / Resource for account creation

Next, create an Entity and Service of ʻAccount` corresponding to the account. First, Entity. Naturally, it is usually a JPA Entity. As a personal hobby, PK uses UUID, but it is possible to use a separate sequence without any problem.

@Entity
public class Account implements Serializable {
    private UUID id;
    private Long amount;

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    public UUID getId() {
        return id;
    }
    public void setId(UUID id) {
        this.id = id;
    }
    public Long getAmount() {
        return amount;
    }
    public void setAmount(Long amount) {
        this.amount = amount;
    }
}

Next is the Service that operates the Entity.

@ApplicationScoped
public class AccountService {
    @Inject
    EntityManager em;

    @Transactional
    public void create(long amount) {
        Account account = new Account();
        account.setAmount(amount);
        em.persist(account);
    }
}

Finally, it is a Resource that describes JAX-RS.

@Path("/account")
public class AccountResource {
    @Inject
    AccountService accountService;

    @POST
    @Produces(MediaType.APPLICATION_JSON)
    @Path("/create")
    public void create() {
        accountService.create(0);
    }
}

Let's hit / account / create.

$ curl -i -X POST -H "Content-Type: application/json" http://localhost:8080/account/create
HTTP/1.1 204 No Content
Date: Mon, 15 Apr 2019 02:56:30 GMT

It ended normally. Also, I think that the SQL executed as follows appears in the standard output of the server. Since I changed quarkus.hibernate-orm.log.sql to true, I can check the SQL log. Don't forget to set it to false in production.

Hibernate: 
    insert 
    into
        Account
        (amount, id) 
    values
        (?, ?)

Add API for creating the rest of the account

Now let's add some crispy and rest of the API. ʻAccountService.java` has a list and deposit / withdrawal function.

@ApplicationScoped
public class AccountService {

    @Inject
    EntityManager em;

    @Transactional
    public void create(long amount) {
        Account account = new Account();
        account.setAmount(amount);
        em.persist(account);
    }

    @Transactional
    public List<Account> findAll() {
        return em.createQuery("SELECT a FROM Account a", Account.class)
                .setMaxResults(3)
                .getResultList();
    }

    @Transactional
    public Account deposit(UUID id, long amount) {
        em.createQuery("UPDATE Account SET amount = amount + :amount WHERE id=:id")
                .setParameter("id", id)
                .setParameter("amount", amount)
                .executeUpdate();
        return em.createQuery("SELECT a FROM Account a WHERE id=:id", Account.class)
                .setParameter("id", id)
                .getSingleResult();
    }

    @Transactional
    public Account withdraw(UUID id, long amount) {
        em.createQuery("UPDATE Account SET amount = amount - :amount WHERE id=:id")
                .setParameter("id", id)
                .setParameter("amount", amount)
                .executeUpdate();
        return em.createQuery("SELECT a FROM Account a WHERE id=:id", Account.class)
                .setParameter("id", id)
                .getSingleResult();
    }
}

Then modify ʻAccountResource.java` as follows.

@Path("/account")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public class AccountResource {

    @Inject
    AccountService accountService;

    @POST
    public void create() {
        accountService.create(0);
    }

    @GET
    public List<Account> list() {
        return accountService.findAll();
    }

    @POST
    @Path("/deposit/{id}/{amount}")
    public Account deposit(@PathParam("id") UUID id, @PathParam("amount") long amount) {
        System.out.println(id + ":" + amount);
        return accountService.deposit(id, amount);
    }

    @POST
    @Path("/withdraw/{id}/{amount}")
    public Account withdraw(@PathParam("id") UUID id, @PathParam("amount") long amount) {
        System.out.println(id + ":" + amount);
        return accountService.withdraw(id, amount);
    }
}

I will try this.

$ curl -X POST -H "Content-Type: application/json" http://localhost:8080/account
$ curl -X GET -H "Content-Type: application/json" http://localhost:8080/account 
[{"amount":0,"id":"0687662d-5ac7-4951-bb11-c9ced6558a40"}]                                               
$ curl -X POST -H "Content-Type: application/json" http://localhost:8080/account/deposit/0687662d-5ac7-4951-bb11-c9ced6558a40/100 
{"amount":100,"id":"0687662d-5ac7-4951-bb11-c9ced6558a40"}                                              
$ curl -X POST -H "Content-Type: application/json" http://localhost:8080/account/deposit/0687662d-5ac7-4951-bb11-c9ced6558a40/100
{"amount":200,"id":"0687662d-5ac7-4951-bb11-c9ced6558a40"}                                                 
$ curl -X POST -H "Content-Type: application/json" http://localhost:8080/account/deposit/0687662d-5ac7-4951-bb11-c9ced6558a40/100
{"amount":300,"id":"0687662d-5ac7-4951-bb11-c9ced6558a40"}                                                  
$ curl -X POST -H "Content-Type: application/json" http://localhost:8080/account/withdraw/0687662d-5ac7-4951-bb11-c9ced6558a40/200
{"amount":100,"id":"0687662d-5ac7-4951-bb11-c9ced6558a40"} 

I was able to implement the deposit / withdrawal function safely.

documentation

Now that the app function is complete, the next step is documentation. However, Quarkus supports OpenAPI and Swagger UI, so it can be done quickly.

First, add Extension.

$ mvn quarkus:list-extensions|grep openapi
[INFO] 	 * SmallRye OpenAPI (io.quarkus:quarkus-smallrye-openapi)
$ mvn quarkus:add-extension -Dextensions="io.quarkus:quarkus-smallrye-openapi"

After adding the Extention, rerun quarkus: dev. Then go to http: // localhost: 8080 / openapi. Then, you can get the OpenAPI definition file generated from JAX-RS as shown below.

openapi: 3.0.1
info:
  title: Generated API
  version: "1.0"
paths:
  /account:
    get:
      responses:
        200:
          description: OK
          content:
            application/json: {}
    post:
...

You can also access the following Swagger UI documents by accessing http: // localhost: 8080 / swagger-ui /. 004.png

Deploy to Cloud Run

Now that the explosive Java EE app has been created by Quarkus, let's deploy it to the Serverless environment Cloud Run. As I wrote at the beginning, Cloud Run is a GCP-based CaaS (Containers as a Service) environment based on Knative. There are ways to deploy to your own GKE environment and a handy GCP fully managed environment, but this time we will use the latter.

Creating Cloud SQL

Since we are using PostgreSQL this time, RDB is also required for GCP. That's why we use Cloud SQL to create a managed DB.

$ gcloud sql instances create myinstance --region us-central1 --cpu=2 --memory=7680MiB --database-version=POSTGRES_9_6
$ gcloud sql users set-password postgres --instance=myinstance --prompt-for-password

You have now created a DB with the name my instance. You can check the operation with gcloud sql instances list.

$ gcloud sql instances list
NAME        DATABASE_VERSION  LOCATION       TIER              PRIMARY_ADDRESS  PRIVATE_ADDRESS  STATUS
myinstance  POSTGRES_9_6      us-central1-b  db-custom-2-7680  xxx.xxx.xxx.xxx   -                RUNNABLE

Creating a Docker image for Cloud Run

Next, create a Docker image for Cloud Run. Originally, no special settings are required for Cloud Run. Images built with any of the src / main / docker / Dockerfile.native | jvm included in the Quarkus project will work as is. However, GCP-managed Cloud Run cannot be placed inside a VPC (Cloud Run on GKE is possible). Therefore, the connection from Cloud Run to Cloud SQL is based on Cloud SQL Proxy instead of connecting directly with ACL.

Create endpoint script

Create a src / main / script / run.sh script that runs both the Proxy and Quarkus apps, such as:

#!/bin/sh

# Start the proxy
/usr/local/bin/cloud_sql_proxy -instances=$CLOUDSQL_INSTANCE=tcp:5432 -credential_file=$CLOUDSQL_CREDENTIALS &

# wait for the proxy to spin up
sleep 10

# Start the server
./application -Dquarkus.http.host=0.0.0.0 -Dquarkus.http.port=$PORT

It takes some time to start Proxy, so I put Sleep for about 10 seconds. Obviously, it takes at least 10 seconds to spin up, so I want to do something about it ...

Creating and building a Dockerfile

Next, create a Dockerfile. The basics are the same as Dockerfile.native, but create src / main / docker / Dockerfile.gcp including run.sh and cloud_sql_proxy.

FROM registry.fedoraproject.org/fedora-minimal
WORKDIR /work/
COPY target/*-runner /work/application
RUN chmod 775 /work

ADD https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 /usr/local/bin/cloud_sql_proxy
RUN chmod +x /usr/local/bin/cloud_sql_proxy
COPY src/main/script/run.sh /work/run.sh
RUN chmod +x /work/run.sh

EXPOSE $PORT
CMD ["./run.sh"]

Then add Dockerfile.gcp to .dockerignore. In the Quarkus project, only specific files are hidden at build time, so if you do not modify it, an error will occur as there is no target file at docker build. The repair differences are as follows.

$ diff .dockerignore.bak .dockerignore
4a5
> !src/main/script/run.s

Now that we're ready, let's build. In the fully managed version of Cloud Run, it seems that the deployment target needs to be located in Cloud Registry, so set the tag to gcr.io/project name / image name.

$ export PRJ_NAME="Project name here"
$ ./mvnw clean package -Pnative -DskipTests=true -Dnative-image.container-runtime=docker
$ docker build -f src/main/docker/Dockerfile.gcp -t gcr.io/${PRJ_NAME}/mybank .

GraalVM's native-image build is very heavy as it checks for dependencies and converts to a native image. It takes 5 to 10 minutes, so please enjoy your nostalgic coffee time.

Local operation check

To check the operation locally, create an account that can access the SQL client from "IAM and Management" and create a JSON format key. Place this in an appropriate local location with the name credentials.json, place it in a place where you can see the container on the volume, and specify the file path in the environment variable CLOUDSQL_CREDENTIALS to check the operation locally. I will.

$ export SQL_CONNECTION_NAME=$(gcloud sql instances describe myinstance|grep connectionName|cut -d" " -f2)
$ docker run -it -p 8080:8080 -v `pwd`:/key/ \
-e CLOUDSQL_INSTANCE=${SQL_CONNECTION_NAME} \
-e CLOUDSQL_CREDENTIALS=/key/credentials.json \
-e QUARKUS_DATASOURCE_URL=jdbc:postgresql://localhost:5432/postgres \
-e QUARKUS_DATASOURCE_PASSWORD="Password here" \
gcr.io/${PRJ_NAME}/mybank

Add SQL client permissions to your Cloud Run service account

Now that we have confirmed the operation locally, we will make settings for Cloud Run. If you put the key file in the container, it will work as above, but it is sensitive to security, so SQL to the Cloud Run service account Google Cloud Run Service Agent ([email protected]) Add client privileges.

In "IAM and Management"-> "IAM", specify the above Cloud Run Service Agent and add the role of Cloud SQL Client. You can now connect from Cloud Run without a key file.

Deploy to Cloud Run

Now it's time to deploy to Cloud Run. First, push the image you created earlier to Cloud Registry.

docker push gcr.io/${PRJ_NAME}/mybank

Then deploy.

$ export SQL_CONNECTION_NAME=$(gcloud sql instances describe myinstance|grep connectionName|cut -d" " -f2)
$ gcloud beta run deploy mybank \ 
--image gcr.io/${PRJ_NAME}/mybank \
--set-env-vars \
QUARKUS_DATASOURCE_URL=jdbc:postgresql://localhost:5432/postgres,\
QUARKUS_DATASOURCE_PASSWORD={DB password},\
CLOUDSQL_INSTANCE=$SQL_CONNECTION_NAME

You can specify environment variables with set-env-vars. It is convenient to be able to change the log level and DB settings as needed without modifying the module.

$ gcloud beta run services list
   SERVICE         REGION       LATEST REVISION     SERVING REVISION    LAST DEPLOYED BY    LAST DEPLOYED AT
✔  mybank          us-central1  mybank-00017        mybank-00017        [email protected]      2019-04-25T08:50:28.872Z

You can see that it was deployed. Let's check the URL of the connection destination.

$ gcloud beta run services describe mybank|grep hostname
hostname: https://mybank-xxx-uc.a.run.app

Let's check the operation with curl.

[$ curl -X POST -H "Content-Type: application/json" https://mybank-xxx-uc.a.run.app/account
$ curl https://mybank-xxx-uc.a.run.app/account
[{"amount":0,"id":"b9efbb84-3b4d-4152-be6b-2cc68bfcbe71"}]

The first spin-up takes about 10 seconds, but after that you can see that it operates in about several hundred ms. DB access is perfect!

Summary

I created a Serverless Java EE environment using Quarkus and Cloud Run. GAE locks in and Java EE doesn't spin up, so how about a product that feels just right?

Isn't it interesting to start with ms order while writing the Java EE familiar writing style such as JAX-RS / CDI and JPA? This feature fits very well with the Serverless architecture of "launching a process on every request". It seems strange to return to Fast CGI, but I think it is an interesting feature that GC, which bothers Java, does not have a big effect because it does not operate for a long time in principle.

Both of them have just come out and are unstable, and since they are breakthroughs, we need to think about what to do with the operational aspects, but I would like to continue to follow them. First of all, if you do not do something around the DB, it will work in ms order, but the spin-up is heavy due to the DB connection ...

Then Happy Hacking!

reference

Recommended Posts

Serverless Java EE starting with Quarkus and Cloud Run
Get along with Java containers in Cloud Run
Run batch with docker-compose with Java batch
Enable Java EE with NetBeans 9
Run Java VM with WebAssembly
Run Java EE applications on CICS
Use java with MSYS and Cygwin
Distributed tracing with OpenCensus and Java
[Java EE] Implement Client with WebSocket
Use JDBC with Java and Scala.
Run logstash with Docker and try uploading data to Elastic Cloud
Use Java 11 with Google Cloud Functions
Output PDF and TIFF with Java 8
Microservices With Docker and Cloud Performance
Run an application made with Java8 with Java6
Encrypt with Java and decrypt with C #
Java starting from beginner, variables and types
Monitor Java applications with jolokia and hawtio
One-JAR Java EE application with WebSphere Liberty
Link Java and C ++ code with SWIG
Let's try WebSocket with Java and javascript!
[Java] Reading and writing files with OpenCSV
Getting Started with Java Starting from 0 Part 1
AWS Lambda with Java starting now Part 1
Java development with Codenvy: Hello World! Run
Create Java applications with IBM Cloud Functions
Create an ARM-cpu environment with qemu on mac and run java [Result → Failure]
Be careful with requests and responses when using the Serverless Framework in Java
Run Rust from Java with JNA (Java Native Access)
Build and test Java + Gradle applications with Wercker
Try to link Ruby and Java with Dapr
JSON with Java and Jackson Part 2 XSS measures
Build an E2E test environment with Selenium (Java)
Using Gradle with VS Code, build Java → run
Create jupyter notebook with Docker and run ruby
Compile and run Java on the command line
Prepare a scraping environment with Docker and Java
KMS) Envelope encryption with openssl and java decryption
Java starting with JShell-A peek into the Java world
[Java] Convert and import file values with OpenCSV
[Review] Reading and writing files with java (JDK6)