[PYTHON] Delete DynamoDB data after 5 minutes with AWS Step Functions

Introduction

I am developing a web API with API Gateway + Lambda + DynamoDB. I had to write data to DynamoDB with a Lambda function and delete that data after 5 minutes. Therefore, I considered the following method. -Delete the data after 5 minutes using the TTL function of DyanamoDB -Call Lambda to delete data after 5 minutes from Lambda to write data

DynamoDB TTL function

First, I checked the TTL function of DynamoDB, but there was a description that TTL will delete items that have expired within 48 hours from the expiration date, and it was not possible to set to always delete after 5 minutes. https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/howitworks-ttl.html

Use AWS Step Functions

Next, consider calling a Lambda that deletes data from the Lambda that writes the data. As a result of investigation, it seems that it can be realized by using AWS Step Functions. I will implement it.

Creating a Lambda function to write data

The runtime grants access to Python 3.8, DynamoDB and Step Functions. Write data to DynamoDB with the following Lambda function and call StateMachine. Pass userId as a parameter to StateMachine.

lambda_function.py

import  boto3

client = boto3.client('stepfunctions')
dynamoDB = boto3.resource('dynamodb')

def lambda_handler(event, context):
    #Table to write data
    table = dynamoDB.Table('test')
    #Call State Machine
    stateMachineArn = 'State Machine ARN'
    #Write data to DynamoDB
    table.put_item(Item={'data': data})
    #Call StateMachine
    input = {"parameters":{"userId":userId}}
    client.start_execution(
        stateMachineArn = stateMachineArn,
        input = json.dumps(input)
        )
    return {'statusCode': 200, 'body': "writeData"}

Creating a State Machine

Create a State Machine with the following definition. Give your State Machine access to Lambda.

{
  "StartAt": "wait_5_miutes",
  "States": {
    "wait_5_miutes": {
      "Type": "Wait",
      "Seconds": 300,
      "Next": "deleteData"
    },
    "deleteData": {
      "Type": "Task",
      "Resource": "ARN of the Lambda function that deletes the data",
      "Parameters": {
          "userId.$": "$.parameters.userId"
      },
      "End": true
    }
  }
}

Creating a Lambda function that deletes data

Delete the data from StateMachine by executing the following Lambda function.

lambda_function.py

import boto3

dynamoDB = boto3.resource('dynamodb')
table = dynamoDB.Table('test')

#Called by StateMachine
def lambda_handler(event, context):
    #Receive parameters from StateMachine
    userId = event['userId']
    #Find the data to be deleted
    searchResult = table.get_item(Key={'userId': userId})
    if "Item" in searchResult:
        table.delete_item(Key={'userId': userId})
    return {'statusCode': 200, 'body': "deleteData"}

in conclusion

By using AWS Step Functions, we were able to realize write data to DynamoDB and delete that data after 5 minutes. I'd like to try it because it seems that more complicated processing can be done.

reference

https://qiita.com/ketancho/items/147a141c9f8a6de86c97 https://aws.amazon.com/jp/getting-started/hands-on/create-a-serverless-workflow-step-functions-lambda/