I tried to visualize the access of Lambda → Athena with AWS X-Ray

I made something like this before, but I experimented to see what I could understand by visualizing it with AWS X-Ray.

I tried to operate Amazon Athena from AWS Lambda

Running the AWS X-Ray daemon

The AWS X-Ray SDK doesn't seem to send Trace data directly to AWS X-Ray, so create an EC2 instance for sending. It's easy because you just register the following as user data and create an instance.

#!/bin/bash
curl https://s3.dualstack.us-east-1.amazonaws.com/aws-xray-assets.us-east-1/xray-daemon/aws-xray-daemon-2.x.rpm -o /home/ec2-user/xray.rpm
yum install -y /home/ec2-user/xray.rpm

The xray installation log was output to the system log, so it should be OK.

Examining /home/ec2-user/xray.rpm: xray-2.0.0-1.x86_64
Marking /home/ec2-user/xray.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package xray.x86_64 0:2.0.0-1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package         Arch              Version               Repository        Size
================================================================================
Installing:
 xray            x86_64            2.0.0-1               /xray            6.6 M

Transaction Summary
================================================================================
Install  1 Package

Total size: 6.6 M
Installed size: 6.6 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : xray-2.0.0-1.x86_64                                          1/1 
xray start/running, process 2576
  Verifying  : xray-2.0.0-1.x86_64                                          1/1 

Installed:
  xray.x86_64 0:2.0.0-1                                                         

Complete!

Preparation on the Lambda app side

I am running a Java application this time, but if you want to monitor the Lambda application with X-Ray, it seems that you only need to select the following check box in the "Settings" tab of the Lambda application.

スクリーンショット 2017-04-23 22.21.44.png

Reference: http://docs.aws.amazon.com/ja_jp/xray/latest/devguide/xray-services.html

You also need to set permissions in IAM to operate X-Ray. Since this was a trial operation, I added "AWS Xray Full Access", but I would like to carefully select this area according to the actual operation.

Start the app and try to visualize it

If you can do so far, you can run the Lambda app normally and see what it looks like in X-Ray. This time, I gave the following JSON as input to the Lambda application. It is an input value when data is fetched from Athena's table as a sample in the previous article.

{
  "region": "us-east-1",
  "s3Path": "s3://ishida-athena-staging-dir/",
  "sql": "SELECT elbname, requestip,  requestport, backendip, backendport, requestprocessingtime, backendprocessingtime, timestamp FROM sampledb.elb_logs order by timestamp desc limit 10",
  "columnListStr": "elbname, requestip,  requestport, backendip, backendport, requestprocessingtime, backendprocessingtime,  timestamp"
}

After waiting for about 1 minute after execution, the following display was confirmed on X-Ray. It seems that the visualization was successful.

スクリーンショット 2017-04-23 22.56.40.png

Check the contents of X-Ray

When I clicked on the object on the right side of the displayed Service Map, the following was displayed. スクリーンショット 2017-04-23 22.56.51.png

You can see in a list how long each process takes and what is returned as a response. Clicking on the displayed ID confirmed the details of that Trace.

スクリーンショット 2017-04-23 22.56.58.png

As far as I can see, it takes about 230ms to initialize the Lambda application, and about 3 seconds to the actual Athena connection part. This process takes 4.6 seconds as a whole, so it should be understood that it takes about 1.5 seconds other than actually accessing Athena. I need more study in this area (^^;

By the way, if there is an error, it seems that you can also check the contents of the exception.

Summary

How long each process takes, and it is powerful to be able to visualize while setting up the calling relationship so easily. Considering that it will become commonplace to distribute processing with Microservices etc. from now on, it can be said that it is an indispensable technology. Speaking of Spring, it feels like Zipkin and Sleuth are realized on AWS.

Recommended Posts

I tried to visualize the access of Lambda → Athena with AWS X-Ray
I tried to solve the problem of "multi-stage selection" with Ruby
I tried to build the environment of PlantUML Server with Docker
I tried to check the operation of gRPC server with grpcurl
I tried to measure and compare the speed of GraalVM with JMH
I tried to compare the infrastructure technology of engineers these days with cooking.
I tried to summarize the state transition of docker
05. I tried to stub the source of Spring Boot
I tried to reduce the capacity of Spring Boot
I tried to check the operation of http request (Put) with Talented API Tester
[Beginner's point of view] I tried to solve the FizzBuzz problem "easily" with Ruby!
I tried to investigate the mechanism of Emscripten by using it with the Sudoku solver
First AWS Lambda (I tried to see what kind of environment it works in)
I tried to increase the processing speed with spiritual engineering
[JDBC] I tried to access the SQLite3 database from Java.
I tried to summarize the basics of kotlin and java
I tried to build the environment of WSL2 + Docker + VSCode
I tried upgrading from CentOS 6.5 to CentOS 7 with the upgrade tool
I tried to interact with Java
I tried to explain the method
I tried the input / output type of Java Lambda ~ Map edition ~
I tried to summarize the methods of Java String and StringBuilder
I tried to solve the problem of Google Tech Dev Guide
I tried to express the result of before and after of Date class with a number line
When I try to use the AWS SDK with Ruby + Lambda, `sam local` is messed up.
I tried to take a look at the flow of Android development environment construction with Android Studio
I tried to summarize the methods used
I tried to summarize Java lambda expressions
I tried to get started with WebAssembly
I tried to implement the Iterator pattern
I tried to summarize the Stream API
I tried to implement ModanShogi with Kinx
I tried to summarize the key points of gRPC design and development
How to access Socket directly with the TCP function of Spring Integration
I tried to solve the tribonacci sequence problem in Ruby, with recursion.
I tried to make full use of the CPU core in Ruby
After all I wanted to preview the contents of mysql with Docker ...
I tried to create a portfolio with AWS, Docker, CircleCI, Laravel [with reference link]
I want to output the day of the week
I tried to verify AdoptOpenJDK 11 (11.0.2) with Docker image
I want to control the start / stop of servers and databases with Alexa
I tried to manage struts configuration with Coggle
I tried to implement a function equivalent to Felica Lite with HCE-F of Android
[Rails] I tried to raise the Rails version from 5.0 to 5.2
I tried to manage login information with JMX
I tried to organize the session in Rails
What I tried when I wanted to get all the fields of a bean
How to use Java framework with AWS Lambda! ??
I tried to chew C # (basic of encapsulation)
I want to var_dump the contents of the intent
I tried to get the distance from the address string to the nearest station with ruby
I tried to set tomcat to run the Servlet.
I tried to clone a web application full of bugs with Spring Boot
[Beginner] I tried to decorate the bar after displaying the details of the hamburger menu
I tried using the profiler of IntelliJ IDEA
I tried to break a block with java (1)
I checked the number of taxis with Ruby
Sazae-san's rock-paper-scissors I tried to verify the theoretical value and the measured value of the probability of the same hand 5 consecutive times with Ruby
I tried to make a product price comparison tool of Amazon around the world with Java, Amazon Product Advertising API, Currency API (2017/01/29)
I tried to summarize the points to consider when acquiring location information with the iOS application ③
I tried to create a log reproduction script at the time of apt install