[PYTHON] I tried to introduce a serverless chatbot linked with Rakuten API to Teams

Introduction

This time I created a serverless chatbot that can search books and introduced it to Teams.

When users enter a keyword in Teams, we've created a chatbot that displays five book titles and prices related to the keyword. In this article, I'll show you how to bring a serverless chatbot to Teams.

background

I will explain the background that led me to create a chatbot.

We were looking for a new initiative aimed at developing new graduates. While searching, I came up with the idea, "Why don't you try pair programming with new graduates and senior employees to create an app?"

The purpose of this proposal is the following two.

――If you use the technical skills of senior employees, you can make things in a short period of time. ――It will stimulate the motivation for learning for new graduates. ――By pair programming with senior employees, you can get various knowledge such as thinking circuit when writing code. ――For new graduates who have little coding experience, it is a valuable experience to know what senior employees are thinking about when coding.

I came up with this chatbot with the flow of trying to work on this idea with me, a new graduate, and senior employees.

By the way, I used Live Share for pair programming. Live Share is a pair programming tool available in VS Code. You can visually grasp where the people you are working with are operating, and since the movement is not heavy, you can comfortably perform pair programming.

For more information on Live Share, see Pair-Pro with Visual Studio Live Share (quoted from Developers.IO).

Configuration diagram of the created chatbot

The processing contents of the created chatbot are as follows.

1. Send keywords in Teams.
2. Send Webhook [^ 1] is activated by sending a keyword as a trigger.
3. The sending webhook packs the keywords into the request Body and makes a POST request to API Gateway.
4. The Lambda function is triggered by the request to API Gateway.
5. Make a GET request to Rakuten API in the Lambda function.
6. Rakuten API searches for books.
7. Respond the search results to Teams.

[^ 1]: Outgoing webhook is a function of Teams that makes a POST request to Teams and external apps.
For more information, see Add custom bots to Microsoft Teams using outbound webhooks (https://docs.microsoft.com/ja-jp/microsoftteams/platform/webhooks-and-connectors/how-to/add-outgoing-webhook) (quoted from the Microsoft Office Developer Center).

Rakuten API uses Rakuten Books Comprehensive Search API. For details on Rakuten Books Comprehensive Search API Please refer to Rakuten Books Comprehensive Search API (version: 2017-04-04) (quoted from Rakuten Developers).

Below is the configuration diagram.

Creation procedure

1. Preparing the AWS environment

This time we will mainly use two AWS services.

• AWS Lambda --Lambda function is executed triggered by a specific event.
  We will realize a serverless application that can execute functions without installing a server.
• Amazon API Gateway --Use API Gateway to trigger Lambda.
  API Gateway is a fully managed service on AWS that makes it easy to create APIs.

The Lambda function is implemented in python. The implementation contents will be introduced in 3. Script description (using python).

Set the Lambda function to be executed when a POST request is made to the URL of API Gateway. Please refer to API Creation Tutorial with API Gateway and Lambda (quoted from @vankobe) for the setting method.

2. Preparation of Rakuten API

To use the Rakuten API, you need to register as a Rakuten member and issue an app ID. Please refer to How to use Rakuten Product Search API-Find the lowest priced product (PHP)- (quoted from HP code) for how to issue the app ID.

Issuing an app ID and using Rakuten API is free.

3. Script writing (using python)

To implement the process, I created the following three functions.

1. A function that searches for books with keywords entered by the user Function that extracts only the title and price parameters from the processing result of the function in 2.1 A function that calls the functions of 3.1 and 2 and describes a series of chatbot processes.

3-1. Function to search for books by keywords entered by the user

This function searches for books by the keyword entered by the user.

When the function is called, it receives the keyword entered by the user as an argument.

In addition, the search result of the book is received from Rakuten API with json and returned with dict type.

Below is a python script.

REQUEST_URL = "https://app.rakuten.co.jp/services/api/BooksTotal/Search/20170404"
APP_ID="Enter the issued app ID here"
def search_books(keyword):
    serch_params={
    "applicationId" : [APP_ID],
    "formatVersion": 2,
    "keyword" : keyword,
    "hits": 5,
    "availability": 0,
    "sort": "reviewAverage"
    "booksGenreId": "001"
    }
    response = requests.get(REQUEST_URL, serch_params)
    result = response.json()
    return result

Store the input parameters of Rakuten API in search_params. This time, I specified the following parameters.

• formatVersion --Specify the output format version.
• hits --Specify the number of acquisitions per page.
• availability --Specify the stock status of books in the search results.
• sort --Specify the order in which the search results are sorted.
• booksGenreId --Specify the genre in Rakuten Books.

For other input parameters, refer to Rakuten Books Comprehensive Search API Input Parameters version: 2017-04-04 (quoted from Rakuten Developers).

Function that extracts only the title and price parameters from the processing result of the 3-2.1 function

This function extracts only the title and price parameters from the search results processed by function 1.

When the function is called, the processing result of the function 1 is received as an argument.

In addition, the result extracted by the title and price parameters is returned as a list type.

Below is a python script.

def extract_books(result):
    item_list = [{'title': item['title'], 'price': item['price']} for item in result['Items']]
    return item_list

Loop around the book obtained by function 1 and store the result extracted by the title and price parameters in item_list as a list type. In addition, the above processing uses the inclusion notation.

A function that calls the functions 3-3.1 and 2 and describes a series of chatbot processes.

In this function, 1 and 2 functions are called to execute a series of processing required for chatbot and respond to API Gateway.

Below is a python script.

app = Chalice(app_name='Enter the API name here')
def search():
    json_text = app.current_request.json_body['text']
    keyword = json_text[json_text.find('</at>')+5:]
    keyword = html.unescape(keyword)
    result = search_books(keyword)
    text = ''
    for item in extract_books(result):
        text += f'**title**： {item["title"]}。\n\n**price**： {item["price"]}Circle\n\n------------------------------------------\n\n'
    return {'text':text}

Store the keywords entered in Teams in json_text. The content of the request from Teams to API Gateway is described with reference to CloudWatch Logs. (Chalice used to store app is explained in 4. Deploy python script to Lambda.)

When sending a keyword from Teams to API Gateway, a specific string is escaped with an html string, so The keyword is stored in keyword after setting not to escape using the unescape function.

After storing the desired information in list type in result, the process to convert list type to String type is performed.

Finally, if you return the dict type object [^ 2] whose key is set in text, the processing of the python script is completed.

[^ 2]: When responding to Teams, you need to include text in the key. If you do not include the
key, it will not be treated as text and an error will occur.

3-4. All python scripts

All the scripts required for this chatbot are listed below.

import requests
import html
from chalice import Chalice

app = Chalice(app_name='Enter the API name here')

REQUEST_URL = "https://app.rakuten.co.jp/services/api/BooksTotal/Search/20170404"
APP_ID="Enter the issued app ID here"

@app.route('/bookSearch', methods = ['POST'])
def search():
    json_text = app.current_request.json_body['text']
    keyword = json_text[json_text.find('</at>')+5:]
    keyword = html.unescape(keyword)
    result = search_books(keyword)
    text = ''
    for item in extract_books(result):
        text += f'**title**： {item["title"]}。\n\n**price**： {item["price"]}Circle\n\n------------------------------------------\n\n'
    return {'text':text}

def search_books(keyword):
    serch_params={
    "applicationId" : [APP_ID],
    "formatVersion": 2,
    "keyword" : keyword,
    "hits": 5,
    "availability": 0,
    "sort": "reviewAverage"
    "booksGenreId": "001"
    }
    response = requests.get(REQUEST_URL, serch_params)
    result = response.json()
    return result

def extract_books(result):
    item_list = [{'title': item['title'], 'price': item['price']} for item in result['Items']]
    return item_list

4. Deploy the python script to Lambda

I deployed to Lambda using AWS Chalice.

chalice is an AWS application framework that allows you to deploy code to Lambda with a single command. For more information on AWS Chalice, see What is AWS Chalice? (https://aws.amazon.com/jp/builders-flash/202003/chalice-api/) (quoted from builders.flash).

Here's how to associate your Lambda with the script you're deploying:

from chalice import Chalice
app = Chalice(app_name='Enter the API name here')
@app.route('/bookSearch', methods = ['POST'])
#Describe the processing code below

In the terminal after writing the above script

chalice deploy

You can deploy to Lambda by typing the command.

Please note that AWS Chalice requires an AWS credential file.

5. Operation check

After creating the chatbot, install it in Teams.

See below for how to deploy chatbot on Teams. AWS Lambda (Python), Amazon API Gateway and bot using Microsoft Teams Outgoing Webhooks (quoted from Yamamugi)

When you install the app on your Teams team, you will be able to launch chatbots from all channels that belong to your team.

In the actual channel, mention the chatbot app name and enter a keyword to display the book search results.

Also, if you look at the logs displayed in CloudWatch Logs, you can see that you are searching with the keywords you entered.

Finally

Until now, I've only used AWS services for the purpose of setting up EC2 or using ELB to distribute the load. Also, I was a beginner engineer who had only touched Java as the language.

It was a very valuable experience for me to be able to create one app using API Gateway, Lambda, Python, and external API for the first time.

By pair programming with senior employees, I was able to learn about redundant coding that I would not notice and how to code various patterns. In addition, since senior employees give advice, it is possible to avoid the phenomenon that young people, including new graduates, tend to fall into when coding does not proceed at all after worrying or investigating for a long time.

I realized that pair programming is a very meaningful educational method for the education of new graduates including myself. We encourage you to try pair programming as well.

I had a hard time suddenly using services such as python environment construction (using pipenv) and AWS Chalice that I had never even lectured on, but I learned a lot from this effort. Also, I realized that I could easily create an app by using AWS or an external API. Next, I would like to create an app by myself even if there are no senior employees.

I would like everyone to refer to this article and create various chatbots.