Lambda Function (python version) that decompresses and outputs elements to CloudWatch Logs when a compressed file is uploaded to s3

For studying Lambda. Extract the gz file uploaded to the s3 bucket and output it to CloudWatch Logs.

analyze_s3logs


from __future__ import print_function

import json
import urllib
import boto3

import gzip

print('Loading function')

s3 = boto3.client('s3')

def lambda_handler(event, context):
    print("[LambdaLog] Received event: " + json.dumps(event, indent=2))

    # Get the object from the event and show its content type
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = urllib.unquote_plus(event['Records'][0]['s3']['object']['key']).decode('utf8')
    try:
        s3.download_file(bucket, key, '/tmp/file.dat')

        if ('.gz' in key):
            f = gzip.open('/tmp/file.dat', 'rb')
        else:
            f = open('/tmp/file.dat', 'r')
        content = f.read()
        f.close
        print(content)

        return 0
    except Exception as e:
        print(e)
        print('[LambdaLog] Error getting object {} from bucket {}. Make sure they exist and your bucket is in the same region as this function.'.format(key, bucket))
        raise e

Recommended Posts

Lambda Function (python version) that decompresses and outputs elements to CloudWatch Logs when a compressed file is uploaded to s3
[Python] Regularly export from CloudWatch Logs to S3 with Lambda
[Python] Convert CSV file uploaded to S3 to JSON file with AWS Lambda
Move CloudWatch logs to S3 on a regular basis with Lambda
A story that I fixed when I got Lambda logs from Cloudwatch Logs
I tried to develop a Formatter that outputs Python logs in JSON
Output timing is incorrect when standard (error) output is converted to a file in Python
[Python] Solution to the problem that elements are linked when copying a list
Export Python3 version OpenCV KeyPoint to a file
A story that got stuck when trying to upgrade the Python version on GCE
When writing to a csv file with python, a story that I made a mistake and did not meet the delivery date
Let's use AWS Lambda to create a mechanism to notify slack when the value monitored by CloudWatch is exceeded on Python
Try creating a compressed file using Python and zlib
[C / C ++] Pass the value calculated in C / C ++ to a python function to execute the process, and use that value in C / C ++.
A Python script that saves a clipboard (GTK) image to a file.
Output CloudWatch Logs to S3 with AWS Lambda (Pythyon ver)
Create code that outputs "A and pretending B" in python
How to write a metaclass that supports both python2 and python3
Write a script in Shell and Python to notify you in Slack when the process is finished
Process the gzip file UNLOADed with Redshift with Python of Lambda, gzip it again and upload it to S3