Regularly notify Slack of articles stocked on Qiita with AWS Lambda + DynamoDB + CloudWatch

Overview

--I want to learn by touching AWS services ――I want to notify Slack of Qiita's "articles stocked this week" that will be notified by email every Saturday.

What to use

Overall view

↑ is summarized as follows.

agenda

[0. Preparation](# 0-Preparation) [1. Create template](# 1-Create template) [2. Deploy using SAM](# 2-Deploy using sam) [3. Notify Slack](# 3-Notify slack) [4. Notify Slack of stock information using Qiita API](Notify slack of stock information using # 4-qiita-api) [5. Manage notification history using DynamoDB](# 5-Manage notification history using dynamodb) [6. Make Lambda run periodically at any time in CloudWatch](# 6-Make lambda run regularly at any time in cloudwatch)

0. Advance preparation

AWS

If the following is not created or installed, do it as appropriate.

--Create AWS account -** AWS CLI ** installation --Install AWS CLI version 1 on macOS --AWS Command Line Interface -** Installing AWS SAM CLI ** --[Installing AWS SAM CLI on macOS --AWS Serverless Application Model](https://docs.aws.amazon.com/ja_jp/serverless-application-model/latest/developerguide/serverless-sam-cli-install- mac.html)

Slack

Install the webhook app on the channel you want to be notified and check the operation. Reference: Slack Webhook URL acquisition procedure --Qiita

Implementation

1. Create a template

Create a template with the sam command.

$ sam init -n qiita-stocks-notifier -r ruby2.5

Reference: sam init --AWS serverless application model

I chose the default for the questions asked in the console. When the command is completed, a template with the following structure is created under the directory.

$ tree
.
├── Gemfile
├── README.md
├── events
│   └── event.json
├── hello_world
│   ├── Gemfile
│   └── app.rb
├── template.yaml
└── tests
    └── unit
        └── test_handler.rb

The contents of ** hello_world / app.rb ** only output a message.

2. Deploy using SAM

Now that the template has been created, let's deploy it on Lambda. Reference: [Tutorial: Deploying Hello World Applications-AWS Serverless Application Model](https://docs.aws.amazon.com/ja_jp/serverless-application-model/latest/developerguide/serverless-getting-started-hello- world.html)

$ sam build
$ sam deploy --guided
#I will answer the question.

When the deployment is completed normally, the character string is output to the console and you can confirm that it is flowing.

 {"message": "hello world"}

Confirm that the deployed application is reflected in the management screen of Lambda.

3. Notify Slack

Add slack-notifier to ** hello_world / Gemfile ** and ** bundle install **. stevenosloan/slack-notifier: A simple wrapper for posting to slack channels

gem "slack-notifier"

$ bundle

Initialize ʻapp.rb by passing the Hooks URL to slack-notifier` as shown below, and try to send the test wording.

app.rb

require "json"
require "slack-notifier"

def lambda_handler(event:, context:)
  notifier = Slack::Notifier.new "https://hooks.slack.com/services/**/**" do
    defaults channel: "#qiita-slacks-notification"
  end
  notifier.ping "yeah"
end

3.1 Deploy and check operation

After the modification is completed, build and deploy it to check the operation.

$ sam build
$ sam deploy

On the Lambda administration screen, select "Select Test Event"-> "Set Test Event".

It will be a screen to select the parameters for the created application, but this time the behavior does not change depending on the parameters. Leave the default.

When you're done setting up test events, select Test and verify that the message is sent to Slack.

Now you can confirm the communication from ** Lambda ** to Slack.

4. Notify Slack of stock information using Qiita API

Next, get the user's stock from Qiita API and notify it.

4.1 Check API documentation

Here is the API to get the articles stocked by the user. Qiita API v2 documentation-Qiita: Developer

Include a ** Faraday ** gem to make it easier to interact with external APIs. Reference: lostisland / faraday: Simple, but flexible HTTP client library, with support for multiple backends.

gem "faraday"

$ bundle

4.2 Implementation of notification part

The outline of the implementation part is as follows.

-Get stock information from API using ** Faraday ** --Notify Slack of responses in an array by ** title ** and ** URL **

app.rb

require 'json'
require 'faraday'
require 'slack-notifier'

USER_ID = "{My user ID}".freeze

def lambda_handler(event:, context:)
  #Get stock information from API
  response = Faraday.get("https://qiita/api/v2/user/#{USER_ID}/stocks", page: 1, per_page: 10)

  #Arrange titles and URLs in an array
  messages = JSON.parse(response.body).each_with_object([]) do |res, ary|
    ary << [res["title"], res["url"]]
  end

  notifier = Slack::Notifier.new "https://hooks.slack.com/services/**/**" do
    defaults channel: "#qiita-slacks-notification"
  end

  notifier.ping "I will publish the article stocked on Qiita last week"
  messages.each do |message|
    notifier.ping message[0]
    notifier.ping message[1]
    notifier.ping "- - - - - - - - - - - - - - - - - - - - "
  end
end

Once the fix is complete, try using sam local invoke, which allows you to test your app locally.

sam local invoke --no-event

Reference: [sam local invoke --AWS serverless application model](https://docs.aws.amazon.com/ja_jp/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-local- invoke.html)

You can confirm that the stock information has been sent to Slack by executing the command.

5. Manage notification history using DynamoDB

The notification part has been completed, but if it is left as it is, the same information may flow every time, so I would like to prevent the article once notified from being notified. Use ** DynamoDB ** to register notification information and prevent notification of information in the database.

Use ** ʻaws-record` ** gem to exchange data with DynamoDB. aws/aws-sdk-ruby-record: Official repository for the aws-record gem, an abstraction for Amazon DynamoDB.

5.1 Implementation

template.yml

AWSTemplateFormatVersion: '2010-09-09'
...
Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: ruby2.5
      Policies:
+     #Create to DynamoDB/Read/Update/Grant permission to delete
+     - DynamoDBCrudPolicy:
+         TableName: !Ref NotifiedMessageDDBTable
+     #Environment variable
+     Environment:
+       Variables:
+         DDB_TABLE: !Ref NotifiedMessageDDBTable
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get
+ NotifiedMessageDDBTable:
+   Type: AWS::Serverless::SimpleTable
...

app.rb

require 'aws-record'
require 'json'
require 'faraday'
require 'slack-notifier'

USER_ID = "{My user ID}".freeze

class NotifiedMessage
  include Aws::Record
  set_table_name ENV['DDB_TABLE']
  string_attr :id, hash_key: true
  string_attr :title
  string_attr :url
end

def lambda_handler(event:, context:)
  #Get notified information
  already_notified_messages = []
  NotifiedMessage.scan.each do |notified_message|
    already_notified_messages << notified_message.url
  end

  #Get stocked articles from API and put only unannounced ones in an array
  response = Faraday.get("https://qiita.com/api/v2/users/#{USER_ID}/stocks", page: 1, per_page: 10)
  messages = JSON.parse(response.body).each_with_object([]) do |res, ary|
    ary << [res["title"], res["url"]] unless already_notified_messages.include? res["url"]
  end

  notifier = Slack::Notifier.new "https://hooks.slack.com/services/xx/xx" do
    defaults channel: "#qiita-stocks-notification"
  end
  notifier.ping "I didn't have any articles in stock this week" and return if messages.size.zero?

  #Notify Slack and save to database
  notifier.ping "I will publish the article stocked on Qiita last week"
  messages.each do |message|
    notifier.ping message[0]
    notifier.ping message[1]
    notifier.ping "- - - - - - - - - - - - - - - - - - - -"

    notified_message = NotifiedMessage.new(id: SecureRandom.uuid, title: message["title"], url: message["url"])
    notified_message.save!
  end
end

If you deploy and test this, you can check the added table on the DynamoDB management screen and insert the latest 10 stock information.

6. Make CloudWatch run Lambda at any time

Finally, make sure you get notifications to Slack at a nice time each week.

6.1 Schedule settings

If you select ** Add trigger ** and select" ** CloudWatch Events / Event Bridge ** "from the search, you can set the rules for execution, so I set it as follows.

The screenshot was leaked, but the ** schedule expression ** was set correctly with cron (0 12? * SAT *) (every Saturday at 12:00). Reference: Schedule expression using Rate or Cron --AWS Lambda

If you check the scheduler from CloudWatch, you can see that it runs ** every Saturday at 12:00 ** as shown below.

Summary

You can now receive weekly stock notifications from Qiita. However, if it is left as it is, the number of readings of DynamoDB will increase and the wallet will be pinched, so it is necessary to narrow down the information to be acquired by passing query to scan. I learned a lot from AWS services because I often understand them only by touching them.

I would appreciate it if you could point out any mistakes.

Reference article

Official documentation

-sam init --AWS serverless application model -[sam local invoke --AWS serverless application model](https://docs.aws.amazon.com/ja_jp/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-local-invoke .html) -Amazon DynamoDB example --AWS SDK for Ruby

The blog that I used as a reference

-AWS Lambda now supports Ruby, so it's super easy to try with SlackBot --Qiita -Register data from Lambda to DynamoDB with Ruby --Toranoana Development Office Blog