What is Pynamodb

A wrapper library for boto3. Writing in boto3 makes things complicated. In Documentation, Pynamodb supports such things! Is written.

But why stop there? PynamoDB also supports:

· Sets for Binary, Number, and Unicode attributes ・ Automatic pagination for bulk operations ・ Global secondary indexes ・ Local secondary indexes ・ Complex queries

While reading Library Contents, I will interpret each item in my own way.

Sets for Binary, Number, and Unicode attributes You can define various attributes such as Binary, Number, Unicode and so on.

class UserModel(Model, User):
    class Meta:
        table_name = "User"
        region = "ap-northeast-1"
        billing_mode = "PAY_PER_REQUEST"
    
    user_id = UnicodeAttribute(hash_key=True)
    execution_date_time = UnicodeAttribute(range_key=True)
    target_month = UnicodeAttribute()
    charge = NumberAttribute(null=True)
    user_type = UnicodeEnumAttribute(UserType)
    secondary_index = SecondaryIndex()

Looking at attributes.py, Pynamodb internally serializes it to change the attribute type to what DynamoDB allows. On the contrary, by deserializing, the data stored in DynamoDB is converted to the type used in the model.

If you want to use a type other than Type provided by Pynamodb, you need to create your own in Custom Attribute.

For those who are troublesome, there is also a convenient library called pynamodb-attributes, so you may use this. By the way, the above UnicodeEnumAttribute uses this library.

Automatic pagination for bulk operations You can get a large amount of data over 1MB in a simple way.

Since Dynamodb can only acquire data up to 1MB at a time, when scanning and querying with boto3, for example, write as follows. Quoted from Official Document You check the LastEvalutedKey of Response, and if necessary, add it to the request and get it again.

done = False
start_key = None
while not done:
    if start_key:
       scan_kwargs['ExclusiveStartKey'] = start_key
    response = table.scan(**scan_kwargs)
    display_movies(response.get('Items', []))
    start_key = response.get('LastEvaluatedKey', None)
    done = start_key is None

Since the amount of description is large, there may be some people who have created their own wrapper class. This kind of description is no longer necessary in Pynamodb (it is done automatically inside Pynamodb).

In Pynamodb, the return type of scan and query is an iterator called ResultIterator. There is an implementation in pagination.py, but here the special method \ __next \ __ assigns LastEvaluatedKey to ExclusiveStartKey. This makes it possible to automatically acquire a large amount of data.

Also, since the result is returned by an iterator, the memory consumption is less than when operating with boto3.

Secondary indexes ・ Global secondary indexes ・ Local secondary indexes Both can be indexed in a way similar to the model definition of a table. There is a difference whether the inheritance source is LocalSecondaryIndex or GlobalSecondaryIndex.


class SecondaryIndex(LocalSecondaryIndex):
   class Meta:
       index_name = "secondaryIndex"
       projection = AllProjection()
   
   id = UnicodeAttribute(hash_key=True)
   target_month = UnicodeAttribute(range_key=True)

Queries are also simple. [Model class]. [Index name defined in the model class] .query ().

UserModel.secondary_index.query(target_month)

Complex queries You can simply write complex queries using expressions. The arguments that can be included in the query are in here and can be used, for example:


filter_condition = (
            UserModel.user_type == user_type
            if user_type
            else None
        )

return list(
            UserModel.query(
                hash_key=id,
                range_key_condition=range_key_condition,
                filter_condition=filter_condition,
                consistent_read=False,
                scan_index_forward=False,
            )
        )

The arguments that are often used are as follows. hash_key: Hash key specification range_key_condition: Specify range key filter_condition: Specify with attributes other than hash key and range key consistent_read: Whether to read strongly consistently scan_index_forward: Sort in ascending order (True) or sort in descending order (False)

By the way, it is not limited to Pynamodb, but since filter_condition filters items after querying, please note that the size of Response does not decrease even if the result of this filter becomes 0. The amount of communication is also determined according to the number of cases before the filter.

It is often used in filter_condition, but there is a way to write conditional expressions in Condition Expressions. You can also use this for conditional operations.

info = UserModel(id="abc",name="Taro")
info.save(UserModel.id.exists())

bonus

It was interesting to Implement the comparison operator used in Condition Expressions while using a special method of python. Reading the library is a learning experience, so I definitely wanted to continue.

[PYTHON] I searched about Pynamodb

What is Pynamodb

bonus