AWS Lambda says Container Image Support, so pseudo Cloud Run
Introduction
Merry Christmas! !! : christmas_tree:: christmas_tree:: santa:: christmas_tree :: christmas_tree:
The 25th day of Ateam Lifestyle Advent Calendar 2020 Ateam Lifestyle Inc. Executive Officer CTO @tsutorm is in charge!
I heard that Lambda supports container images
What should I do with this year's ad-care: thinking:
It will be a poem system, but let's write something like "development productivity" of engineers who can easily make mistakes in business.
Christmas: christamas_tree: That's right. It's better to have a code that will be a gift
Where I thought ...
[Breaking News] Container images are now supported as a Lambda package format! !! #reinvent
I thought it was Cloud Ru ..., but is it a little different?
-Try Lambda Container Support with AWS SAM CLI -Explosive speed? !! I tried to verify the cold start of Lambda deployed from the container image #reinvent
There are already some articles that I touched (everyone is early), but it seems that it is not Cloud Run as I expected.
――I thought it was a Cloud Run-like guy with API Gateway on Lambda. -BaseImage is like Amazon Linux2, so if you create a Dockerfile properly, Rails will work in principle. --There are Aurora Serverless and RDS Proxy, and I think I can make Web API.
...
Let's try it together!
Rails API on AWS Lambda Run (provisional) It works
$ #Please do something like REST or seeds because Rails is used for data stuffing.
$ curl https://xxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/posts # GET /posts
[{"id":1,"title":"test","body":"hoge","published":null,"created_at":"2020-12-23T11:31:39.626Z","updated_at":"2020-12-23T11:31:39.626Z"},{"id":2,"title":"test2","body":"hoge","published":null,"created_at":"2020-12-23T12:11:04.785Z","updated_at":"2020-12-23T12:11:04.785Z"},{"id":3,"title":"test3","body":"hoge","published":null,"created_at":"2020-12-23T13:29:04.307Z","updated_at":"2020-12-23T13:29:04.307Z"}]
START RequestId: 87bfb3ea-568d-477a-aa52-809fc14c3d40 Version: $LATEST
I, [2020-12-24T06:35:23.744874 #8] INFO -- : [de7866af-0a31-4bd7-a7f1-9b15aeb864dc] Started GET "/posts" for 64.252.172.83 at 2020-12-24 06:35:23 +0000
I, [2020-12-24T06:35:23.746365 #8] INFO -- : [de7866af-0a31-4bd7-a7f1-9b15aeb864dc] Processing by PostsController#index as */*
I, [2020-12-24T06:35:23.960053 #8] INFO -- : [de7866af-0a31-4bd7-a7f1-9b15aeb864dc] Completed 200 OK in 214ms (Views: 212.8ms | ActiveRecord: 27.8ms | Allocations: 6188)
END RequestId: 87bfb3ea-568d-477a-aa52-809fc14c3d40
REPORT RequestId: 87bfb3ea-568d-477a-aa52-809fc14c3d40 Duration: 220.86 ms Billed Duration: 3193 ms Memory Size: 1024 MB Max Memory Used: 158 MB Init Duration: 2971.41 ms
Overall feeling
Overall view of composition
It's not as complicated as it sounds, but it looks like this in the figure
Only that
code
As an application, it's a very simple one that just adds a certain post model to something like rails new --api, so what happens if you fit an existing Rails application that you made tightly? I think that will be another problem. ..
I threw everything around deploy to serverless. Very convenient.
The docker build is partly shelled, but the official documentation didn't seem too difficult.
I did only the construction of Aurora Serverless manually, but since it is not such a difficult story, I will omit the procedure explanation here.
build & deploy
I'm sorry I couldn't script it completely, but it's about below
--build to create an image ./build.sh
--Enter the information required for deploy, including image hash, in dev.yml
serverless deploy--Invoke the Lambda function equivalent torails db: migratevia CLI only for the first time
$ aws lambda invoke --function-name lambda-run-dev-migrate --payload '{}' response.json # migration
Where I was addicted, where I stumbled
Dockerfile.lambda
I don't know the choice of my image including Base Image or Ruby runtime interface clients provided by AWS.
-Official document It's a story that can be read properly. ――At first I wrote a Dockerfile based on Alpine, but I don't like having to put aws_lambda_ric because it was invaded by Gemfile. Recreated with official Base Image --AWS-provided BaseImage is only Ruby 2.5/2.7
FROM public.ecr.aws/lambda/ruby:2.7.2020.12.18.21
BaseImage has aws_lambda_ric specified in advance in entrypoint, so if you do docker run -it public.ecr.aws/lambda/ruby: 2.7/bin/sh, it will not accept input: thinking:
before
$ docker run --rm -it public.ecr.aws/lambda/ruby:2.7 /bin/sh
INFO[0000] exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)
#like this
after
$ docker run --rm --entrypoint="" -it public.ecr.aws/lambda/ruby:2.7 /bin/sh
sh-4.2#
I'm writing Dockerfile while doing yum install and playing with Areya Koreya
serverless
Basically, only I referred to it because there was a very easy-to-understand entry
Overwriting of CMD and ENTRYPOINT is not yet supported
Avoid by preparing another function to rails db: migrate with another image
# Dockerfile.lambda
#Abbreviation
FROM app as migrate
WORKDIR /var/task
#Omission
CMD ["lambda.App::Handler.migrate"]
# lambda.rb
#Abbreviation
def self.migrate(event:, context:)
p `bin/rails db:create`
p `bin/rails db:migrate`
response = {
'statusCode' => 204
}
end
end
If you want to connect with Aurora, you have to use VPC Lambda
I completely forgot and made it normally, so I remade it.
It was serverless, so it's easy with serverless remove && serverless deploy
# serverless.yml
vpc:
securityGroupIds:
- ${self:custom.environment.${self:provider.stage}.MY_AWS_SECURITY_GROUP}
subnetIds:
- ${self:custom.environment.${self:provider.stage}.MY_AWS_SUBNET_ID_0}
- ${self:custom.environment.${self:provider.stage}.MY_AWS_SUBNET_ID_1}
- ${self:custom.environment.${self:provider.stage}.MY_AWS_SUBNET_ID_2}
iamRoleStatements:
- Effect: "Allow"
Action:
- "ec2:CreateNetworkInterface"
- "ec2:DescribeNetworkInterfaces"
- "ec2:DeleteNetworkInterface"
Resource:
- "*"
Rails on Lambda
How do you get Rails running from Lambda in the first place? ??
I tried Ruby on Rails on AWS Lambda and API Gateway by Serverless Framework was saved because I had a predecessor. Almost this. Thank you very much.
It works with aws_lambda_ric → lambda.rb → config.ru.
The code passed from the handler to rack changed only the reference path of config.ru based on the following.
https://github.com/aws-samples/serverless-sinatra-sample/blob/master/lambda.rb
Therefore, Rails does not start puma, which is a common application server:: exclamation:: exclamation:
As a result, I feel that it has the benefit of reducing the puma startup overhead that was expected to occur during invoke.
Scales from the eyes. Botamochi from the shelf.
Since / var/task / of WORKDIR is write-protected, all the processing to write to Rails.root.join ('tmp') is an error
The main thing I was addicted to this time was the cache file in bootsnap and tmp/log/*. Log
bootsnap cache cannot be written to tmp/cache
Avoid using environment variables by changing the output destination of the bootsnap cache to / tmp as per the bootsnap documentation
# Dockerfile.lambda
ENV BOOTSNAP_CACHE_DIR=/tmp/cache
Changed to output log to STDOUT because log cannot be written to tmp/log/development.log
I'm having trouble getting on cloudwatch logs normally, so I just do this, and there is a story that it was RAILS_ENV = development at that time. You should also write it in config/environments/development.rb
# config/environments/development.rb
if ENV["RAILS_LOG_TO_STDOUT"].present?
logger = ActiveSupport::Logger.new(STDOUT)
logger.formatter = config.log_formatter
config.logger = ActiveSupport::TaggedLogging.new(logger)
end
# Dockerfile.lambda
ENV RAILS_LOG_TO_STDOUT=1
How it feels when loaded
Resource status when only one request is thrown
cold start
$ hay -c 1 -n 1 https://xxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/posts
Summary:
Total: 3.7599 secs
Slowest: 3.7597 secs
Fastest: 3.7597 secs
Average: 3.7597 secs
Requests/sec: 0.2660
Total data: 408 bytes
Size/request: 408 bytes
Response time histogram:
3.760 [1] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
3.760 [0] |
3.760 [0] |
3.760 [0] |
3.760 [0] |
3.760 [0] |
3.760 [0] |
3.760 [0] |
3.760 [0] |
3.760 [0] |
3.760 [0] |
Latency distribution:
0% in 0.0000 secs
0% in 0.0000 secs
0% in 0.0000 secs
0% in 0.0000 secs
0% in 0.0000 secs
0% in 0.0000 secs
0% in 0.0000 secs
Details (average, fastest, slowest):
DNS+dialup: 0.1575 secs, 3.7597 secs, 3.7597 secs
DNS-lookup: 0.1406 secs, 0.1406 secs, 0.1406 secs
req write: 0.0001 secs, 0.0001 secs, 0.0001 secs
resp wait: 3.6014 secs, 3.6014 secs, 3.6014 secs
resp read: 0.0006 secs, 0.0006 secs, 0.0006 secs
Status code distribution:
[200] 1 responses
Warm start
$ hay -c 1 -n 1 https://xxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/posts
Summary:
Total: 0.0556 secs
Slowest: 0.0556 secs
Fastest: 0.0556 secs
Average: 0.0556 secs
Requests/sec: 17.9856
Total data: 408 bytes
Size/request: 408 bytes
Response time histogram:
0.056 [1] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.056 [0] |
0.056 [0] |
0.056 [0] |
0.056 [0] |
0.056 [0] |
0.056 [0] |
0.056 [0] |
0.056 [0] |
0.056 [0] |
0.056 [0] |
Latency distribution:
0% in 0.0000 secs
0% in 0.0000 secs
0% in 0.0000 secs
0% in 0.0000 secs
0% in 0.0000 secs
0% in 0.0000 secs
0% in 0.0000 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0159 secs, 0.0556 secs, 0.0556 secs
DNS-lookup: 0.0052 secs, 0.0052 secs, 0.0052 secs
req write: 0.0003 secs, 0.0003 secs, 0.0003 secs
resp wait: 0.0386 secs, 0.0386 secs, 0.0386 secs
resp read: 0.0005 secs, 0.0005 secs, 0.0005 secs
Status code distribution:
[200] 1 responses
Aurora works with capacity 1
When loaded (300 parallel / 3000 requests)
$ hay -c 300 -n 3000 https://xxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/posts
Summary:
Total: 7.3692 secs
Slowest: 5.1843 secs
Fastest: 0.0223 secs
Average: 0.4378 secs
Requests/sec: 407.1016
Total data: 370236 bytes
Size/request: 123 bytes
Response time histogram:
0.022 [1] |
0.538 [2645] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
1.055 [59] |■
1.571 [3] |
2.087 [0] |
2.603 [2] |
3.120 [5] |
3.636 [20] |
4.152 [255] |■■■■
4.668 [9] |
5.184 [1] |
Latency distribution:
10% in 0.0351 secs
25% in 0.0404 secs
50% in 0.0490 secs
75% in 0.0728 secs
90% in 0.8516 secs
95% in 3.8275 secs
99% in 4.0212 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0331 secs, 0.0223 secs, 5.1843 secs
DNS-lookup: 0.0079 secs, 0.0000 secs, 0.1565 secs
req write: 0.0009 secs, 0.0000 secs, 0.2680 secs
resp wait: 0.3979 secs, 0.0222 secs, 4.7219 secs
resp read: 0.0004 secs, 0.0000 secs, 0.1758 secs
Status code distribution:
[200] 907 responses
[500] 2093 responses
Don't be afraid. The cause is insufficient capacity on the RDS side, so if you stand for a certain period of time ...
$ hay -c 300 -n 3000 https://xxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/posts
Summary:
Total: 1.3416 secs
Slowest: 0.7870 secs
Fastest: 0.0215 secs
Average: 0.1167 secs
Requests/sec: 2236.2066
Total data: 844968 bytes
Size/request: 281 bytes
(Abbreviation)
Status code distribution:
[200] 2071 responses
[500] 929 responses
Aurora is also serverless, so the capacity increases without permission and it can be handled without permission! !! !!
After a certain amount of time ...
I'll scale in towards zero
Where I couldn't touch
ActiveJob (sidekiq) will also be Lambda
Cost comparison with ECS Fargate/EKS
Lambda should be bad in the case of commercial use where there is a certain workload above a certain level. I think.
Summary
――I'm glad it was fun. -I feel good with Cloud Run. I'm fully aware of the story There are also Jets. I have no regrets. ――I don't know if it's a huge Rails. Insufficient verification.
in conclusion
This is the end of Ateam Lifestyle Advent Calendar 2020. How was that?
Thank you @engabesi, @asasigure for moving to the operation!
2020 was a turbulent year, mainly with Covid-19.
What kind of year will 2021 be next year? I'm looking forward to it!
Then everyone Happy Holidays !!!: tada: