Try implementing asynchronous processing in Azure

ALH Advent Calender This is ALH Advent Calender! I participated in the Nori. Introducing secretly

This time, I will introduce the architecture in Azure in a hands-on format.

It's premised on rudimentary knowledge in Azure, so I'm sorry for Hatsumi-san, but I'm sorry: cry:

Asynchronous architecture on Azure

I would like to use Azure Service Bus (https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-overview), which is a managed messaging service. ..

Why asynchronous? about,

--Separate heavy processing --Improved scalability --Load leveling --Buffering

There are merits such as.

There are patterns that use Azure Queue Storage and DB polling as options for the architecture of online delay, but I chose Service Bus because it seems easy to implement. (Rather, I want to use it)

Messaging is a niche area such as MQ, but I think it is an indispensable architecture for cloud-native applications.

Although it is in English, we recommend that you take a look below. Enterprise Integration Patterns

• The table of contents is on the right.

Input in the previous stage

Architectural best practices are detailed in the official Azure documentation.

I also recommend other chapters because you can study just by reading them personally!

[Azure] Improving the scalability of Azure web applications [Azure] Queue-based load leveling pattern

The opportunity to write

Java-based documentation and samples in Azure are tough

The hands-on materials are quite substantial, but if you try to implement it with a little in-depth content, there are many C # and Node.js documents, and Java implementation is difficult. It is necessary to select a language that suits the environment.

However, as of now (as of 12/07/2019), Pivotal seems to be working hard.

https://pivotal.io/azure-spring-cloud

In addition, the SDK for Azure Storage has the following problems, and v12 has preview: innocent: There is a GA myth, but when using it in an actual project, preview is really scary, isn't it?

Microsoft Azure Storage SDK for Java Don't use v10,11

I hope this post will be helpful to anyone!

Diagram

It has the following configuration.

ServiceBus can be roughly divided into queues and topics, and JMS and MQ can be used, but this time we will create them in queues.

https://www.draw.io/ I used this to create the figure. Convenient!

Preparing ServiceBus resources

You can create it on the portal, but this time I will create it with CLI.

Execution environment

• macOS Mojave
• Azure CLI 2.0.68
• JDK Zulu 1.8.0_232
• Azure Functions Core Tools 2.7.1948

#Azure login
az login

#Create resource group
az group create --name async-test --location japaneast

#ServiceBus namespace creation (resource allocation) about 1 minute
az servicebus namespace create --resource-group async-test --name serbicebusqueue --location japaneast --sku Basic

#Creating a queue
az servicebus queue create --resource-group async-test --namespace-name serbicebusqueue --name funcｑueue

Regarding --name at the time of create, it is necessary to be unique on Azure, so if the following error occurs, change it to another name.

#When the set name cannot be used
BadRequest - The specified name is not available. CorrelationId: XXXXXXXXXXXXXXXXXXXXXXX

The portal sets $ RANDOM. https://docs.microsoft.com/ja-jp/azure/service-bus-messaging/service-bus-quickstart-cli

Execute the following command, and if the connection string is displayed normally, the creation is complete.

az servicebus namespace authorization-rule keys list --resource-group async-test --namespace-name serbicebusqueue --name RootManageSharedAccessKey --query primaryConnectionString --output tsv

Endpoint=sb://serbicebusqueue.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=[Random string]

Although it is the basics of the cloud, please be careful when handling access keys.

Creating WebApps

Immediately get a mounting sample. First is the implementation of the sender to ServiceBus.

Create a blank project with Spring Boot Initializr

https://start.spring.io/

--Check Web and Azure Support

Add dependency

pom.xml

		<!-- for Azure Service Bus -->
		<dependency>
			<groupId>com.microsoft.azure</groupId>
			<artifactId>azure-servicebus-spring-boot-starter</artifactId>
		</dependency>

Setting application.properties

The connection setting to ServiceBus is set by the Configuration class in starter, so set the connection string and queue of ServiceBus created earlier.

application.properties

#Service Bus settings
azure.servicebus.connection-string=Endpoint=sb://serbicebusqueue.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=[Random string]
azure.servicebus.queue-name=funcqueue
azure.servicebus.queue-receive-mode=peeklock

Creating a Service class

Following the MVC model, implement it as a Service class called from Controller.

EnqueueService.java

package com.example.async.domain;

import com.microsoft.azure.servicebus.Message;
import com.microsoft.azure.servicebus.QueueClient;
import com.microsoft.azure.servicebus.primitives.ServiceBusException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.nio.charset.StandardCharsets;

@Service
public class EnqueueService {

    private static final Logger logger = LoggerFactory.getLogger(EnqueueService.class);

    /**
     *AutoConfigure hides the connection settings to ServiceBus.
     */
    @Autowired
    QueueClient queueClient;

    public boolean sendQueueMessage() {
        String messageBody = "queue Message";
        logger.info("message body:{}", messageBody);
        //Generate message to send
        Message message = new Message(messageBody.getBytes(StandardCharsets.UTF_8));
        try {
            //Send to Service Bus
            queueClient.send(message);
            queueClient.close();
        } catch (ServiceBusException e) {
            e.printStackTrace();
            return false;
        } catch (InterruptedException e) {
            e.printStackTrace();
            return false;
        }
        return true;
    }
}

Creating a Controller

Prepare the endpoint from the client and call the Service class to send to ServiceBus.

AsyncController.java

package com.example.async.controller;

import com.example.async.domain.EnqueueService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("async")
public class AsyncController {

    @Autowired
    EnqueueService enqueueService;

    @PostMapping
    public String sendMessage() {
        boolean result = enqueueService.sendQueueMessage();
        if(!result) {
            return "Fail.";
        }
        return "Complete.";
    }
}

Local execution

At this point, make sure that it is sent locally to ServiceBus.

mvn spring-boot:run

Try hitting curl from another console.

curl -X POST localhost:8080/async

If it is working properly, you can check it on the portal.

Creating Functions

Now that we have created the sender, we will implement the receiver. Use the trigger function of Functions.

I tried to implement it in Spring Cloud Function, but I can't use the Functions extension well and I implement it in pure Java. I would like to re-verify it at a later date: sob:

Create project from maven archetype

Create a blank project from the Maven archetype according to the official documentation. Since it will be in interactive mode, I set it as follows.

mvn archetype:generate \
    -DarchetypeGroupId=com.microsoft.azure \
    -DarchetypeArtifactId=azure-functions-archetype 

[abridgement]

Define value for property 'groupId' (should match expression '[A-Za-z0-9_\-\.]+'): com.example.function
[INFO] Using property: groupId = com.example.function
Define value for property 'artifactId' (should match expression '[A-Za-z0-9_\-\.]+'): functions
[INFO] Using property: artifactId = functions
Define value for property 'version' 1.0-SNAPSHOT: : 0.0.1-SNAPSHOT
Define value for property 'package' com.example.function: : 
Define value for property 'appName' functions-20191208134240755: : async-functions
Define value for property 'appRegion' westus: : japaneast
Define value for property 'resourceGroup' java-functions-group: : async-test
Confirm properties configuration:
groupId: com.example.function
groupId: com.example.function
artifactId: functions
artifactId: functions
version: 0.0.1-SNAPSHOT
package: com.example.function
appName: async-functions
appRegion: japaneast
resourceGroup: async-test
 Y: : y

Creating a function that triggers ServiceBus

Implement as follows.

ServiceBusFunctions.java

package com.example.function;

import com.microsoft.azure.functions.ExecutionContext;
import com.microsoft.azure.functions.annotation.FunctionName;
import com.microsoft.azure.functions.annotation.ServiceBusQueueTrigger;

/**
 * Azure Functions with ServiceBus Trigger.
 */
public class ServiceBusFunction {

    /**
     *Trigger Service Bus
     * @param message
     * @param context
     */
    @FunctionName("servicebus-trigger")
    public void run(
            @ServiceBusQueueTrigger(name = "message",queueName = "funcqueue", connection="SERVICE_BUS_CONNECTIONSTRING") String message,
            final ExecutionContext context) {
        context.getLogger().info("ServiceBusTrigger start.");
        context.getLogger().info(message);
    }
}

Setting value of @ServiceBusQueueTrigger

local.settings.json settings

By default, a configuration file for the local environment is created.

json:local.settings.json

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "java",
    "SERVICE_BUS_CONNECTIONSTRING": "Endpoint=sb://serbicebusqueue.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=[Key value]"
  }
}

--It is not necessary in the default setting that uses HTTP triggers, but if you want to use other triggers, you need to set ʻAzureWebJobsStorage. If local, set ʻUse Development Storage = true. --Set the Service Bus connection string in SERVICE_BUS_CONNECTIONSTRING set in the class.

Run in local environment

Let's execute Functions locally and get the message sent to ServiceBus by local execution earlier.

mvn clean package
mvn azure-functions:run

Hopefully the following log should be output.

[Omit log for HTTP trigger]

[2019/12/08 5:15:56] Executed 'Functions.servicebus-trigger' (Succeeded, Id=e1c75286-c80f-45d9-88ae-d5e8dfb2cd8a)
[2019/12/08 5:18:52] Executing 'Functions.servicebus-trigger' (Reason='New ServiceBus message detected on 'funcqueue'.', Id=07b50ad5-256e-45ee-8bfa-0fb5ddfb7172)
[2019/12/08 5:18:52] Trigger Details: MessageId: 3c3bffa6-c533-41f3-9bf4-7664e16e44bc, DeliveryCount: 1, EnqueuedTime: 2019/12/08 5:18:52, LockedUntil: 2019/12/08 5:19:52, SessionId: (null)
[2019/12/08 5:18:52] ServiceBusTrigger start.
[2019/12/08 5:18:52] queue Message
[2019/12/08 5:18:52] Function "servicebus-trigger" (Id: 07b50ad5-256e-45ee-8bfa-0fb5ddfb7172) invoked by Java Worker
[2019/12/08 5:18:52] Executed 'Functions.servicebus-trigger' (Succeeded, Id=07b50ad5-256e-45ee-8bfa-0fb5ddfb7172)

You have now confirmed that Local Web App-> Serbvice Bus-> Local Functions can be executed successfully.

Deploy to Azure environment

Deploy the created application to Azure.

You can also create resources with the CLI, but Azure samples are often created automatically at deploy time and contain random strings, so create an AppService in the portal.

Creating an App Service

AppService allocates web app resources. You can save resources by deploying multiple applications on the same App Service. However, since resources such as CPU are shared, it is safer to divide the Functions into a pay-as-you-go plan for actual operation.

I created a resource group for ʻasync-test` when I created the service bus, so I will use that.

As of December 7, 2019, when implementing Java in Functions, Windows is the only OS provided for App Service, so select Windows.

Deploy Web Apps

Add the Maven plugin to pom.xml.

pom.xml

      <plugin>
        <groupId>com.microsoft.azure</groupId>
        <artifactId>azure-webapp-maven-plugin</artifactId>
        <version>1.8.0</version>
        <configuration>
          <schemaVersion>V2</schemaVersion>
          <resourceGroup>async-test</resourceGroup>
          <appName>async-app</appName>
          <appServicePlanName>appservice-async-test</appServicePlanName>
          <region>japaneast</region>
          <!--AppServicePlan plan-->
          <pricingTier>S1</pricingTier>
          <runtime>
            <os>windows</os>
            <javaVersion>1.8</javaVersion>
            <webContainer>java 8</webContainer>
          </runtime>
          <deployment>
            <resources>
              <resource>
                <directory>${project.basedir}/target</directory>
                <includes>
                  <include>*.jar</include>
                </includes>
              </resource>
            </resources>
          </deployment>
        </configuration>
      </plugin>

Deploy with maven's ʻazure-webapp-maven-plugin`.

mvn clean package
mvn azure-webapp:deploy

curl command

curl -H 'Content-Length:0' -X POST https://async-app.azurewebsites.net/async

The Content-Length header is required.

Deploy Functions

Since the template is generated when creating a blank project, set AppServiePlan.

pom.xml

      <plugin>
        <groupId>com.microsoft.azure</groupId>
        <artifactId>azure-functions-maven-plugin</artifactId>
        <configuration>
          <resourceGroup>${functionResourceGroup}</resourceGroup>
          <appName>${functionAppName}</appName>
          <region>${functionAppRegion}</region>
          <!--AppServicePlan settings-->
          <appServicePlanName>appservice-async-test</appServicePlanName>

          <appSettings>
            <!-- Run Azure Function from package file by default -->
            <property>
              <name>WEBSITE_RUN_FROM_PACKAGE</name>
              <value>1</value>
            </property>
            <property>
              <name>FUNCTIONS_EXTENSION_VERSION</name>
              <value>~2</value>
            </property>
          </appSettings>
        </configuration>
        <executions>
          <execution>
            <id>package-functions</id>
            <goals>
              <goal>package</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

Deploy with maven's ʻazure-functions-maven-plugin`.

mvn clean package
mvn azure-functions:deploy

After deploying, the function has been created, so set the connection string of ServiceBus in the application settings.

Sample source

https://github.com/bokuwakuma/azure-spring-boot-servicebus

Summary

I ended up with hands-on Azure documentation, but I'm addicted to Azure Functions in many ways. There are some addictive points if you don't try to make a little more application.

Also, I don't think I will deploy directly from local, so I would like to try deploying in a pipeline using GitHub Actions. Next time, I would like to do more about this.