[PYTHON] Set up a Lambda function and let it work with S3 events!

What is Lambda?

One of the services provided by AWS, it is a service that can execute processing without server management.

For example, if you do not use Lambda, the process will be executed only by creating a server with EC2, installing the middleware and language required for processing execution, and setting the environment.

However, with Lambda, it is possible to execute by just writing the process without provisioning such a server.

This will ★ No need to manage or maintain the server itself ★ Costs can be significantly reduced depending on the frequency of processing because you are charged only for the time when processing is being executed. ★ It is very convenient for AWS main architecture because it can be easily linked with other AWS services. There is a merit.

What is a trigger?

It is necessary to set a trigger to execute the written process. It can be linked with all AWS services, and is typically executed when an alarm occurs in CloudWatch, when data exists in the kinesis data stream, or when a file is placed in S3.

In other words, Lambda is a service that works if you write only ** conditions to be executed ** and ** processes to be executed **.

I want to actually set it!

Now let's upload the file to s3 and write a process to confirm that it has been uploaded.

1. Log in to the AWS console and click "Create Function" on the Lambda screen.

Lambda will set the function for each region. There is no problem with S3, but when linking with services affected by other regions, select the same region. image.png

2. Select "Create from 1" and set "Function name", "Runtime", and "Execution role".

This time I will use python 3.7. The role should be "Create new role with basic Lambda permissions". There is no problem using existing roles. In that case, Lambda must have permission to write the log group to CloudWatch for the trigger condition service and Log output.

"Runtime" means the language of processing. As of 02/05/2020, the languages that can be selected are as follows. ・ Java 11/8 ・ .NET Core2.1 (C # / PowerShell) ・ GO 1.x ・ Because. js 12. x / 10. x ・ Python 3.8 / 3.7 / 3.6 / 2.7

3. Set the execution condition with "Add trigger"

This time I will enable it to be executed when some object is created in s3. When the trigger medium is set to s3, there are the following five items. ·bucket -Event type (if the file is PUT ..., if the file is deleted ... etc.) -Prefix (directory path after bucket, file name, etc.) ・ Suffix (file name, extension, etc.) -Trigger activation (trigger with the above settings will work as soon as you check it. Let's turn it on after the processing test is completed) image.png

4. Set the function code

By default, the process is described in a file called lambda_function.py.

import json

def lambda_handler(event, context):
    # TODO implement
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

lambda_handler is a function that is automatically executed when the trigger condition set in Lambda is met. Edit # TODO implement and after, and write the process you want to execute.

This time I will try to display the path and file name where the file is placed in s3.

import json

def lambda_handler(event, context):
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = event['Records'][0]['s3']['object']['key']
 print (bucket +'+ key +'has been created in the bucket!')

The event information that triggered the event is contained in an array in `ʻevent`` passed as an argument to lambda_handler. The above gets the bucket name and file path from it and displays it.

5. Execution / confirmation

The function itself can be executed by placing the file in the path specified by the trigger. Besides that, it is possible to execute the Lambda function in a pseudo manner by setting the value that enters `ʻevent`` from the "test" in the image below in json format. tempsnip.png

Also, for the function execution log, you can select the target function from the CloudWatch log group and check it from there. tempsnip.png

I was able to confirm that the Lambda function was executed correctly! !! tempsnip.png

At the end

Lambda uses the AWS environment, and if you want to perform simple processing, it is highly recommended in terms of cost and management. I was a little worried because there is no space to write an explanation of what the function is doing, so let's thoroughly describe the function name and comments in the process so that you can have a comfortable Lambda life! !!

Recommended Posts

Set up a Lambda function and let it work with S3 events!
Associate Python Enum with a function and make it Callable
Steps to set up Pipenv, create a CRUD app with Flask, and containerize it with Docker
Set up a Samba server with Docker
[AWS] Link Lambda and S3 with boto3
[AWS lambda] Deploy including various libraries with lambda (generate a zip with a password and upload it to s3) @ Python
Introduction and usage of Python bottle ・ Try to set up a simple web server with login function
Set up a simple HTTPS server with asyncio
Set up a local server with Go-File upload-
[Piyopiyokai # 1] Let's play with Lambda: Creating a Lambda function
Set up a local server with Go-File download-
Load a photo and make a handwritten sketch. With zoom function. Tried to make it.
I set up TensowFlow and was addicted to it, so make a note
Set up a Python development environment with Sublime Text 2
[Vagrant] Set up a simple API server with python
Set up a Python development environment with Visual Studio Code
Set up a web server with CentOS7 + Anaconda + Django + Apache
Try to bring up a subwindow with PyQt5 and Python
Create a deploy script with fabric and cuisine and reuse it
Process the gzip file UNLOADed with Redshift with Python of Lambda, gzip it again and upload it to S3
Create Cognito user list in S3 with SQS Deploy queue function and API to Lambda with SAM
How to set a shortcut to switch full-width and half-width with IBus
Move CloudWatch logs to S3 on a regular basis with Lambda
Set CloudWatch Events in AWS Lambda function (WebAPI) deployed via zappa
Create a temporary file with django as a zip file and return it
[Python 3.8 ~] How to define a recursive function smartly with a lambda expression
Make a thermometer with Raspberry Pi and make it viewable with a browser Part 4
Set up Ubuntu as a Linux cheat sheet and https server
I made a chatbot with Tensor2Tensor and this time it worked
Get a list of camera parameters that can be set with cv2.VideoCapture and make it a dictionary type
Upload data to s3 of aws with a command and update it, and delete the used data (on the way)