[PYTHON] AWS Step Functions to learn with a sample
Step Functions announced as a workflow service. Other workflow services include Simple Workflow (SWF), but for new applications it is recommended to consider Step Functions. If it doesn't meet your needs, you can try using SWF.
There are currently 6 types of blueprints available as samples in Step functions. Just by looking at this code, it seems that you can understand the minimum specifications required.
Look at the Blueprint prepared first and create a state machine with all kinds of States defined at the end.
Blueprint
Hello World
{
"Comment": "A Hello World example of the Amazon States Language using a Pass state",
"StartAt": "HelloWorld",
"States": {
"HelloWorld": {
"Type": "Pass",
"Result": "Hello World!",
"End": true
}
}
}
Wait State
{
"Comment": "An example of the Amazon States Language using wait states",
"StartAt": "FirstState",
"States": {
"FirstState": {
"Type": "Task",
"Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
"Next": "wait_using_seconds"
},
"wait_using_seconds": {
"Type": "Wait",
"Seconds": 10,
"Next": "wait_using_timestamp"
},
"wait_using_timestamp": {
"Type": "Wait",
"Timestamp": "2015-09-04T01:59:00Z",
"Next": "wait_using_timestamp_path"
},
"wait_using_timestamp_path": {
"Type": "Wait",
"TimestampPath": "$.expirydate",
"Next": "wait_using_seconds_path"
},
"wait_using_seconds_path": {
"Type": "Wait",
"SecondsPath": "$.expiryseconds",
"Next": "FinalState"
},
"FinalState": {
"Type": "Task",
"Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
"End": true
}
}
}
Retry Failure
{
"Comment": "A Retry example of the Amazon States Language using an AWS Lambda Function",
"StartAt": "HelloWorld",
"States": {
"HelloWorld": {
"Type": "Task",
"Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
"Retry": [
{
"ErrorEquals": ["HandledError"],
"IntervalSeconds": 1,
"MaxAttempts": 2,
"BackoffRate": 2.0
},
{
"ErrorEquals": ["States.TaskFailed"],
"IntervalSeconds": 30,
"MaxAttempts": 2,
"BackoffRate": 2.0
},
{
"ErrorEquals": ["States.ALL"],
"IntervalSeconds": 5,
"MaxAttempts": 5,
"BackoffRate": 2.0
}
],
"End": true
}
}
}
Parallel
{
"Comment": "An example of the Amazon States Language using a parallel state to execute two branches at the same time.",
"StartAt": "Parallel",
"States": {
"Parallel": {
"Type": "Parallel",
"Next": "Final State",
"Branches": [
{
"StartAt": "Wait 20s",
"States": {
"Wait 20s": {
"Type": "Wait",
"Seconds": 20,
"End": true
}
}
},
{
"StartAt": "Pass",
"States": {
"Pass": {
"Type": "Pass",
"Next": "Wait 10s"
},
"Wait 10s": {
"Type": "Wait",
"Seconds": 10,
"End": true
}
}
}
]
},
"Final State": {
"Type": "Pass",
"End": true
}
}
}
Catch Failure
{
"Comment": "A Catch example of the Amazon States Language using an AWS Lambda Function",
"StartAt": "HelloWorld",
"States": {
"HelloWorld": {
"Type": "Task",
"Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
"Catch": [
{
"ErrorEquals": ["HandledError"],
"Next": "CustomErrorFallback"
},
{
"ErrorEquals": ["States.TaskFailed"],
"Next": "ReservedTypeFallback"
},
{
"ErrorEquals": ["States.ALL"],
"Next": "CatchAllFallback"
}
],
"End": true
},
"CustomErrorFallback": {
"Type": "Pass",
"Result": "This is a fallback from a custom lambda function exception",
"End": true
},
"ReservedTypeFallback": {
"Type": "Pass",
"Result": "This is a fallback from a reserved error code",
"End": true
},
"CatchAllFallback": {
"Type": "Pass",
"Result": "This is a fallback from a reserved error code",
"End": true
}
}
}
Choice State
{
"Comment": "An example of the Amazon States Language using a choice state.",
"StartAt": "FirstState",
"States": {
"FirstState": {
"Type": "Task",
"Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
"Next": "ChoiceState"
},
"ChoiceState": {
"Type" : "Choice",
"Choices": [
{
"Variable": "$.foo",
"NumericEquals": 1,
"Next": "FirstMatchState"
},
{
"Variable": "$.foo",
"NumericEquals": 2,
"Next": "SecondMatchState"
}
],
"Default": "DefaultState"
},
"FirstMatchState": {
"Type" : "Task",
"Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:OnFirstMatch",
"Next": "NextState"
},
"SecondMatchState": {
"Type" : "Task",
"Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:OnSecondMatch",
"Next": "NextState"
},
"DefaultState": {
"Type": "Fail",
"Cause": "No Matches!"
},
"NextState": {
"Type": "Task",
"Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
"End": true
}
}
}
combine
The following 7 types of states are available.
- Task
- Wait
- Pass
- Succeed
- Fail
- Choice
- Parallel
Create the following state machine using all 7 types of State.
Implementation
JSON The above state machine is defined in JSON as below.
{
"Comment": "Sample Step functions flow",
"StartAt": "Process1",
"States": {
"Process1": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:XXXXXXXXXXXX:function:Process1",
"Next": "ChoiceStep"
},
"ChoiceStep": {
"Type": "Choice",
"Choices": [{
"Variable": "$.processResult",
"StringEquals": "Process2a",
"Next": "SuccessProcess"
}, {
"Variable": "$.processResult",
"StringEquals": "Process2b",
"Next": "Process2b-1"
}]
},
"SuccessProcess": {
"Type": "Succeed"
},
"Process2b-1": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:XXXXXXXXXXXX:function:Process2",
"Next": "Process2b-2"
},
"Process2b-2": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:XXXXXXXXXXXX:function:Process2",
"Catch": [{
"ErrorEquals": [
"HandledError"
],
"Next": "FailedProcess"
}],
"Next": "Process3"
},
"FailedProcess": {
"Type": "Fail"
},
"Process3": {
"Type": "Wait",
"Seconds": 30,
"Next": "Process4"
},
"Process4": {
"Type": "Parallel",
"Next": "Process5",
"Branches": [
{
"StartAt": "Process4a",
"States": {
"Process4a": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:XXXXXXXXXXXX:function:Process2",
"End": true
}
}
},
{
"StartAt": "Process4b-1",
"States": {
"Process4b-1": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:XXXXXXXXXXXX:function:Process2",
"Retry": [{
"ErrorEquals": [
"HandledError"
],
"IntervalSeconds": 3,
"MaxAttempts": 5,
"BackoffRate": 1.5
}],
"Next": "Process4b-2"
},
"Process4b-2": {
"Type": "Pass",
"Result": {
"data1": 12345,
"data2": -12345
},
"ResultPath": "$.test-data",
"Next": "Process4b-3"
},
"Process4b-3": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:XXXXXXXXXXXX:function:Process2",
"End": true
}
}
}
]
},
"Process5": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:XXXXXXXXXXXX:function:Process3",
"End": true
}
}
}
Lambda function
Prepare the following three Labda functions.
- Process1
def lambda_handler(event, context):
event["step"] = 1
event["processResult"] = "Process2b"
return event
- Process2
def lambda_handler(event, context):
event["step"] += 1
return event
- Process3
def lambda_handler(event, context):
print event
return event
Creation procedure
-
Select [Step functions] from the management console.
-
Select Get Started.
-
Enter the following and select [Create State Machine]. Give a name to your state machine: SampleStateMachineWithVariableOfTypes
Enter the JSON from the previous one.
When you press the load button of Preview, the state of the state machine is displayed.
- Enter the following and select [OK].
IAM role for your state machine executions: StatesExecutionRole-us-east-1
- Select New Execution.
- Give the following value as Input.
{
"processResult": "Process2b"
}
- Select Start Execution.
output
Output
output: [{"processResult": "Process2b", "step": 4},{"test-data": {"data1": 12345, "data2": -12345}, "processResult": "Process2b", "step": 5}]
Tips
-You can check the correctness of the syntax with statelint. (I feel that there are times when it is inaccurate.)
--The correctness of JSON syntax can also be confirmed with the jq command.
--You can check the syntax by pressing the refresh button next to Preview.
--There may be an error even after selecting [Create State Machine] where the above syntax check does not apply, so you can check it by pressing it before execution.
--Debugging Lambda creation
It is easy to make trial and error if you can debug with python sample.py as follows.
def lambda_handler(event, context):
event["step"] = 1
event["processResult"] = "Process2b"
return event
if __name__ == "__main__":
lambda_handler("","")
Caution
--I get angry when I define extra tasks
- There is a problem with your AWL definition, please review it and try again --Spelling mistakes of the specified key, etc.
- Transition references an unknown state --There is no transition destination in Next (spelling mistake, State is not defined in scope, etc.)
reference
- http://docs.aws.amazon.com/step-functions/latest/dg/welcome.html
- https://aws.amazon.com/jp/swf/faqs/
- http://qiita.com/naokitomita/items/901e730a83d73bc0baa3
- http://qiita.com/ketancho/items/147a141c9f8a6de86c97
Recommended Posts