Manipulate DynamoDB data with Lambda (Node & Python)

Introduction

Recently, I am exclusively in charge of IoT related projects. Since I am using AWS, I often store data from devices in DynamoDB, acquire that data and visualize it, and Lambda is very active in that case. I mainly use Node.js, and I have implemented it in Python depending on the project, and I thought that I had made it into a template, so I will summarize it.

Put data to DynamoDB

It mainly writes the data received from the device to DynamoDB. Normally, it is necessary to process the data, but since there are various types depending on the device, I will omit it. The important thing here is to specify the Partition Key and Sort Key. If you make a mistake here, you will get an error. (Key information is wrong! Feeling)

Node.js

const AWS = require("aws-sdk");
const dynamoDB = new AWS.DynamoDB.DocumentClient({
  region: "ap-northeast-1" //DynamoDB Region
});

exports.handler = (event, context, callback) => {
  const params = {
    TableName: "table-name" //DynamoDB table name
    Item: {
      "PartitionKey": "your-partition-key-data", //Partition Key data
      "SortKey": "your-sort-key-data", //Sort Key data
      "OtherKey": "your-other-data"  //Other data
    }
  }
  
  //Execution of Put processing to DynamoDB
  dynamoDB.put(params).promise().then((data) => {
    console.log("Put Success");
    callback(null);
  }).catch((err) => {
    console.log(err);
    callback(err);
  });
}

Python

import boto3

def lambda_handler(event, context):
  try:
    dynamoDB = boto3.resource("dynamodb")
    table = dynamoDB.Table("table-name") #DynamoDB table name
    
    #Execution of Put processing to DynamoDB
    table.put_item(
      Item = {
        "PartitionKey": "your-partition-key-data", #Partition Key data
        "SortKey": "your-sort-key-data", #Sort Key data
        "OtherKey": "your-other-data"  #Other data
      }
    )
  except Exception as e:
        print e

Get DynamoDB data (query)

I mainly use query to get the data in DynamoDB. If you have SortKey in DynamoDB as well as PartitionKey, you can set ScanIndexForward to false and specify the number of cases acquired by Limit, and you can acquire a fixed number from the data in descending order (latest). A common pattern is that you want to get the latest XX data sent by a certain Device ID (Partition Key).

Node.js

const AWS = require("aws-sdk");
const dynamoDB = new AWS.DynamoDB.DocumentClient({
  region: "ap-northeast-1" //DynamoDB Region
});

exports.handler = (event, context, callback) => {
  const params = {
    TableName: "table-name" //DynamoDB table name
    KeyConditionExpression: "#PartitionKey = :partition-key-data and #SortKey = :sort-key-data", //Key information to get
    ExpressionAttributeNames: {
      "#PartitionKey": "your-partition-key", //Partition Key attribute name
      "#SortKey": "your-sort-key" //SortKey attribute name
    },
    ExpressionAttributeValues: {
      ":partition-key-data": "your-partition-key-data", //Partition Key name you want to get
      ":sort-key-data": "your-sort-key-data" //SortKey name you want to get
    }
    ScanIndexForward: false //Ascending or descending order(The default is true=ascending order)
    Limit: 1 //Number of data to be acquired
  }
  
  //Execute query processing to DynamoDB
  dynamoDB.query(params).promise().then((data) => {
    console.log(data);
    callback(data);
  }).catch((err) => {
    console.log(err);
    callback(err);
  });
}

Python

import boto3
from boto3.dynamodb.conditions import Key

def lambda_handler(event, context):
  try:
    dynamoDB = boto3.resource("dynamodb")
    table = dynamoDB.Table("table-name") #DynamoDB table name
    
    #Execute query processing to DynamoDB
    queryData = table.query(
      KeyConditionExpression = Key("your-partition-key").eq("your-partition-key-data") & Key("your-sort-key").eq("your-sort-key-data"), #Key information to get
      ScanIndexForward = false, #Ascending or descending order(The default is true=ascending order)
      Limit: 1 #Number of data to be acquired
    )
    return queryData
  except Exception as e:
        print e

at the end

Since DynamoDB often handles data with Put and query, I have summarized two methods. In some cases, it must be Update instead of Put, or scan is used when you want to get all records for the time being. Don't forget to have a policy that allows you to operate DynamoDB properly *** Lambda! *** *** Also, it sometimes occurs when executing query, but sometimes I want to see "There is no data in DynamoDB even though the process seems to be completed without any error." In such a case, suspect a *** timeout. *** By default, it is 3 seconds, so if the number of data fetched by query is large, it will time out. I wrote this to be used in Lambda, but since I am only using the SDK normally, I think that it can be used by modifying the above source code even when handling data from a web application or a device that can load the SDK. ..

As an aside, one of my recent worries is that if I write Node and write Python, the grammar etc. will be confused ...

(Please forgive me if there is a dirty part in the source code ...)

see you!