Use additional Python packages with Serverless Framework (v1.x)

Serverless (2) Advent Calendar This is the article on the 17th day. To be honest, it's not a big story, but ... (^_^;) Since there is not much information on Python in Serverless Framework (v1.x) and I was worried about it myself, I tried this. (Added at 6:17 AM on December 17) Due to another matter, I changed the input data from CSV to JSON.

background

Originally, in our "[IoT Easy Pack](http://dc.ogis-ri.co.jp/iotpack/" IoT Easy Pack ")", the only message broker that can be selected is MQTT, and it must support HTTPS. It was because I couldn't do it. (It can be published even with REST of AWS IoT, but the authentication is too special and this time the target device is a microcomputer chip, so I had to make the API as simple as possible ... ^^;)

Last time, I tried to run the sample as it is with API Gateway with Serverless Framework (v1.1). [I tried using Serverless Framework](http://qiita.com/daikunjp/items/2b8aa35ecd104001f4e5 I tried using Serverless Framework)

This time, I would like to create an API that inserts CSV data PUT into Web API into MySQL.

Web API construction procedure using Serverless Framework

Serverless Framework version upgrade

Since it was updated to v1.3 before I knew it, I will upgrade the created environment [last time](http://qiita.com/daikunjp/items/2b8aa35ecd104001f4e5 I tried using Serverless Framework).

% npm install -g serverless

Check version

% sls --version
1.3.0

Project creation

Create and initialize the project directory

% mkdir iot-api
% cd iot-api
% sls create --template aws-python --name iot-api

Added MySQL library for Python

I have to put the MySQL driver in Python on lambda ... When I look at the manual, it says that pip can be used for Python, so I installed mysql-python with pip normally, but it doesn't work. Apparently, if you put it in the project directory, it will be put in the zip that uploads to lambda as it is, so install it in the project directory by the following procedure.

% pip install mysql-python -t .

Copy Shared Object to current directory

% cp /usr/lib64/mysql/libmysqlclient.so.18 .

Edit serverless.yml

Edit serverless.yml as follows. This time, I also added VPC settings to access RDS MySQL from lambda 【reference】 https://serverless.com/framework/docs/providers/aws/guide/functions/ http://dev.classmethod.jp/etc/serverless-framework-lambda-function-run-in-vpc/

serverless.yml

service: iot-api
provider:
  name: aws
  runtime: python2.7
  vpc:
   securityGroupIds:
     - sg-xxxxxxxx ← Security group ID
   subnetIds:
     - subnet-xxxxxxxx ← Subnet ID
     - subnet-xxxxxxxx ← Subnet ID
  stage: dev
  region: ap-northeast-1
  iamRoleStatements:
  - Effect: "Allow"
    Action:
      - "ec2:CreateNetworkInterface"
      - "ec2:DescribeNetworkInterfaces"
      - "ec2:DeleteNetworkInterface"
    Resource:
      - "*"
functions:
  iotput:
    handler: handler.iotput
    events:
      - http:
          path: iotput
          method: put

Python code

Click here for the code that actually works Since it is for demo, I do not check the arguments at all (^ _ ^;)

handler.py

# -*- coding: utf-8 -*-
import MySQLdb,json

HOST = "Hostname of the RDS to connect to"
DBNAME = "DB name"
DBUSER = "DB username"
PASSWD = "password"
CHARSET = "utf8"

def iotput(event, context):

  # MySQL Connect
    connect = MySQLdb.connect(host=HOST, db=DBNAME, user=DBUSER, passwd=PASSWD, charset=CHARSET)
    connect.cursorclass = MySQLdb.cursors.DictCursor
    cursor = connect.cursor()

  # BODY Data split for CSV
  #  data1 = event['body'].split(",")
  # BODY Data split for JSON
    data1 = json.loads(event['body'])
    data1[0] = data1['data1']
    data1[1] = data1['data2']
    data1[2] = data1['data3']

  # MySQL Data insert
    cursor.execute("insert into sensor01(data1,data2,data3) values(%s,%s,%s)",
        (data1[0],data1[1],data1[2]))
    connect.commit()
    cursor.close()
    connect.close()

    body = {
        "message": "input data is %s %s %s" % (data1[0],data1[1],data1[2]),
    }

    response = {
        "statusCode": 200,
        "body": json.dumps(body)
    }

    return response

Deploy

% sls deploy
Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading service .zip file to S3 (916.94 KB)...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
..............
Serverless: Stack update finished...

Service Information
service: iot-api
stage: dev
region: ap-northeast-1
api keys:
  None
endpoints:
  PUT - https://xxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/iotput
functions:
  iot-api-dev-iotput: arn:aws:lambda:ap-northeast-1:xxxxxxxxxxxxxxx:function:iot-api-dev-iotput

test

Try testing with curl from another machine.

% curl -X PUT  -d '{"data1":"10","data2":"20","data3":"40"}' https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/iotput
{"message": "input data is 10 20 40"}
% curl -X PUT  -d '{"data1":"1.1","data2":"5.4","data3":"2.7"}' https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/iotput
{"message": "input data is 1.1 5.4 2.7"}

Check the result

mysql> select * from sensor01;
+-------+-------+-------+
| data1 | data2 | data3 |
+-------+-------+-------+
|    10 |    20 |    40 |
|   1.1 |   5.4 |   2.7 |
+-------+-------+-------+
2 rows in set (0.01 sec)