This article is a port of my personal blog Practical Cloud Python --SERVERLESS FRAMEWORK & PYTHON How to use environment variables to Qiita. Thank you for visiting Practical Cloud Python.
Abstract
Explains how to manage environment variables that can be enjoyed by stage management and coding.
The source of the example is released, so please see here for the implementation example.
https://github.com/hassaku63/sls-env-example
It has a configuration of Lambda (scheduled)-> SQS-> Lambda (Queue consumer).
The following packages and plug-ins are used.
Serverless Framework
Python
The example source is code that is supposed to be posted to Slack. Since Token and channel name correspond to confidential information, they are described in .env.
The Serverless Framework uses the Serverless Dotenv Plugin to include the definition of the .env file in provider.environment during deployment packaging.
The Dotenv plugin automatically loads an .env file suffixed with the specified stage name. For example, in the dev stage, load .env.dev as the definition of the environment variable (see the section on Automatic Env file name resolution). This function is used for settings that change API Token etc. for each stage.
It is also declared in the environment variable to manage the resource name of SQS and DynamoDB. This is because the queue and table resource names are needed when using the boto3 client from within the code.
https://gist.github.com/hassaku63/0e4e61db60bd48b5d86459ceabb9dd34
provider:
name: aws
runtime: python3.8
profile: ${opt:profile, default}
stage: ${opt:stage, dev}
region: ${opt:region, us-east-1}
timeout: 30
environment:
# To tell lambda function to AWS AccountId by use pseudo-parameters plugin.
# AccountId is used to construct SQS Queue URL
AWS_ACCOUNT_ID: '#{AWS::AccountId}'
# MY_QUEUE_NAME is used to construct Queue URL in lambda.
# And also convenient to reference resource ARN in IAM Policy statement
MY_QUEUE_NAME: ${self:service}-my-queue-${self:provider.stage}
The part marked $ {self: service} -my-queue- $ {self: provider.stage} is the name of the queue used in this source. The service and stage names are included to ensure the uniqueness of the resource names when using multiple deployment stages.
When enqueue from Python to SQS, you need to specify QueueUrl as the target resource identifier. To assemble QueueUrl, you need three things: AWS account ID, AWS region, and QueueName. I want to be able to refer to them in Lambda, so I declare it with an environment variable (Note that only the region needs to be defined by yourself because Lambda automatically sets it as an environment variable by default at runtime. there is not).
There is talk of whether environment variables should be written in .env or serverless.yml, but I think you should judge whether this is information you want to keep secret. If the number of environment variables becomes too large and it becomes redundant, it is better to pick up only those that are not really environment variables and declare them as environment variables. If there are parts that are not essential (such as substrings that make up part of the resource name), I think it's a good idea to define them in the custom section.
If you want to handle environment variables from Python, you should use os.environ. However, this way of writing doesn't work with IDE auto-completion, so there's no way to notice the variable name Typo when coding. It's not a good development experience to notice mistyping of environment variables only after deploying. Also, it feels like it's a violation of DRY to use os.environ each time the number of Lambda handlers increases.
To prevent this, load environment variables and create settings.py to define them as Python constants. All Python code should read environment variables from this module. This will allow you to take advantage of the IDE's auto-completion while preserving DRY and reducing the possibility of mistyping.
https://gist.github.com/hassaku63/0e971fb0823aea561f33db880d0269e4
"""
Declare all environment variables defined in .env.<stage-name> and serverless.yml as python variables.
If you would like to use environment variables in your code, you import this module then reference as constants of python variable.
By use this technique, you are able to use IDE auto-completion effectively when you writing python code.
It is useful to reduce miss-typing of environment variables.
"""
import os
import pathlib
import json
import dotenv
project_root_dir = pathlib.Path(__file__).parent / '../'
dotenv.load_dotenv()
MYAPP_STAGE_NAME = os.environ.get('MYAPP_STAGE_NAME')
stage_env_path = project_root_dir / f'.env.{MYAPP_STAGE_NAME}'
dotenv.load_dotenv(stage_env_path.resolve())
# Slack
SLACK_BOT_TOKEN = os.environ.get('SLACK_BOT_TOKEN')
SLACK_CHANNEL = os.environ.get('SLACK_CHANNEL')
# AWS
AWS_ACCOUNT_ID = os.environ.get('AWS_ACCOUNT_ID')
# Notice: You don't need to define AWS_REGION in .env and serverless.yml
# because of lambda runtime automatically set AWS_REGION env var when execute.
AWS_REGION = os.environ.get('AWS_REGION')
# SQS
MY_QUEUE_NAME = os.environ.get('MY_QUEUE_NAME', '')
At this time, python-dotenv is used to load .env according to the stage. I expect .env to be in the project root. Find the relative directory for settings in Pathlib, use the MYAPP_STAGE_NAME environment variable to determine the stage name, and use that name as the .env suffix.
Both Serverless Framework and Python have additional packages for working with .env. Let's use these to make good use of environment variables.
There may be references to resource names in your code, such as DynamoDB or SQS resource identifiers. Environment variables can also be used in such cases. By defining it as an environment variable, the variable can be reused even when referring to the resource name in the template.
Let's use os.environ to create a module for grouping environment variables into Python constants. You can also use IDE auto-completion, and since the description is centralized in one place, mistyping is less likely to occur and it is easy.
Recommended Posts