Python calling Google Cloud Vision API from LINE BOT via AWS Lambda

This article, And tried to make almost the same thing with python. Only after using LINE BOT, you will realize its fun and potential. Very fun and convenient, this!

I am very curious about what will happen after the trial is over, but first of all, I will implement it ^ ^

handler.py

# coding:utf-8
# !/usr/bin/python

import line
import json


def lambda_handler(event, context):
    print(json.dumps(event, indent=4, separators=(',', ': ')))
    content = event.get("result")[0].get("content") #Actually, there are multiple results. If it is lambda, it is multi-threaded for the time being. (Although you can put it in SQS)
    line.set_return_text(content)
    line.send_to_line(content)

For the time being, the entrance to AWS Lambda. Since you can get json filled with information from LINE in event, create a response from there and send it to the LINE server.

line.py

# coding:utf-8
# !/usr/bin/python

import vision
import json
import os
import requests

CONTENT_TYPE_TEXT = 1  # Text message
CONTENT_TYPE_IMAGE = 2  # Image message
CONTENT_TYPE_VIDEO = 3  # Video message
CONTENT_TYPE_AUDIO = 4  # Audio message
CONTENT_TYPE_LOCATION = 7  # Location message
CONTENT_TYPE_STICKER = 8  # Sticker message
CONTENT_TYPE_CONTACT = 10  # Contact message

LINE_BOT_API_EVENT = 'https://trialbot-api.line.me/v1/events'
LINE_HEADERS = {
    'Content-type': 'application/json; charset=UTF-8',
    'X-Line-ChannelID': 999999999,  # Channel ID
    'X-Line-ChannelSecret': 'hogehoge',  # Channel secre
    'X-Line-Trusted-User-With-ACL': 'hogehoge'  # MID (of Channel)
}

def set_return_text(content):
    content_type = content.get("contentType")
    if content_type == CONTENT_TYPE_TEXT:
        content["text"] = u"'" + content.get("text") + u"'Is that a difficult question?" + os.linesep + \
                          u"I'm good at photography!"
    elif content_type == CONTENT_TYPE_IMAGE:
        image = get_message_content(content)
        content["text"] = vision.get_image_text(image)
    else:
        content["text"] = u"I'm sorry, I'm not sure>_<" + os.linesep + \
                          u"I'm good at photography!"
    content["contentType"] = CONTENT_TYPE_TEXT


def send_to_line(content):
    data = {
        'to': [content.get('from')],
        'toChannel': 1383378250, #FIX
        'eventType': "138311608800106203", #FIX
        'content': content
    };
    r = requests.post(LINE_BOT_API_EVENT, headers=LINE_HEADERS, data=json.dumps(data))
    print(r.content)


def get_message_content(content):
    url = 'https://trialbot-api.line.me/v1/bot/message/%s/content' % content.get("id")
    r = requests.get(url, headers=LINE_HEADERS)
    return r.content

Pack your credentials in LINE_HEADER. It's solid now, but in reality, if you embed the authentication information from API Gateway's requestTemplate or stageVariable, the repository will not be dirty.

When a photo is sent, it gets the photo data from that id and passes it to vision.py to get the response message.

vision.py

# coding:utf-8
# !/usr/bin/python

# Copyright 2016 Google, Inc
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import base64
import os

from googleapiclient import discovery
from oauth2client.service_account import ServiceAccountCredentials

DISCOVERY_URL = 'https://{api}.googleapis.com/$discovery/rest?version={apiVersion}'
GOOGLE_APPLICATION_CREDENTIALS = {
    "type": "service_account" #Various omissions. Get your credentials from the Google Developer Console.
}

def get_image_text(image):
    request = get_vision_service().images().annotate(body={
        'requests': {
            'image': {
                'content': base64.b64encode(image)
            },
            'features': [
                {"type": "FACE_DETECTION", "maxResults": 5},
                {"type": "LABEL_DETECTION", "maxResults": 5},
                {"type": "TEXT_DETECTION", "maxResults": 5},
                {"type": "LANDMARK_DETECTION", "maxResults": 5},
                {"type": "LOGO_DETECTION", "maxResults": 5},
                {"type": "SAFE_SEARCH_DETECTION", "maxResults": 5}
            ],
            'imageContext': {
                'languageHints': [
                    "ja",
                    "en"
                ]
            }
        },
    })
    response = request.execute()

    annotation = response['responses'][0].get('safeSearchAnnotation')
    if annotation.get("adult") == "POSSIBLE" or annotation.get("adult") == "LIKELY" or annotation.get(
            "adult") == "VERY_LIKELY":
        return u"I don't think it should be naughty!"
    if annotation.get("medical") == "POSSIBLE" or annotation.get("medical") == "LIKELY" or annotation.get(
            "adult") == "VERY_LIKELY":
        return u"That kind of thing ..."
    if annotation.get("spoof") == "POSSIBLE" or annotation.get("spoof") == "LIKELY" or annotation.get(
            "adult") == "VERY_LIKELY":
        return u"You can't be scammed! ??"
    if annotation.get("violence") == "POSSIBLE" or annotation.get("violence") == "LIKELY" or annotation.get(
            "adult") == "VERY_LIKELY":
        return u"Oh, violence is no good! Is useless! Gya!"

    text = u""
    annotations = response['responses'][0].get('labelAnnotations')
    if annotations != None:
        text = text + u'I think it's a photo like this.' + os.linesep
        for annotation in annotations:
            text = text + u'[ ' + annotation.get("description") + u' ]' + os.linesep
        text = text + os.linesep

    annotations = response['responses'][0].get('textAnnotations')
    if annotations != None:
        text = text + u"You can see these characters." + os.linesep
        for annotation in annotations:
            text = text + u'[ ' + annotation.get("description") + u' ]' + os.linesep
        text = text + os.linesep

    annotations = response['responses'][0].get('faceAnnotations')
    if annotations != None:
        text = text + str(len(annotations)) + u"I found a person in the picture!" + os.linesep
        count = 1
        for annotation in annotations:
            text = text + str(count) + u'The eyes are'
            if annotation.get("joyLikelihood") == "POSSIBLE" or annotation.get("joyLikelihood") == "LIKELY" or annotation.get("joyLikelihood") == "VERY_LIKELY":
                text = text + u"looks fun!" + os.linesep
            elif annotation.get("sorrowLikelihood") == "POSSIBLE" or annotation.get("sorrowLikelihood") == "LIKELY" or annotation.get("sorrowLikelihood") == "VERY_LIKELY":
                text = text + u"Looks sad ...!" + os.linesep
            elif annotation.get("angerLikelihood") == "POSSIBLE" or annotation.get("angerLikelihood") == "LIKELY" or annotation.get("angerLikelihood") == "VERY_LIKELY":
                text = text + u"Are you angry?" + os.linesep
            elif annotation.get("surpriseLikelihood") == "POSSIBLE" or annotation.get("surpriseLikelihood") == "LIKELY" or annotation.get("surpriseLikelihood") == "VERY_LIKELY":
                text = text + u"I'm surprised!!" + os.linesep
            elif annotation.get("underExposedLikelihood") == "POSSIBLE" or annotation.get("underExposedLikelihood") == "LIKELY" or annotation.get("underExposedLikelihood") == "VERY_LIKELY":
                text = text + u"Is that underexposure?" + os.linesep
            elif annotation.get("blurredLikelihood") == "POSSIBLE" or annotation.get("blurredLikelihood") == "LIKELY" or annotation.get("blurredLikelihood") == "VERY_LIKELY":
                text = text + u"It's out of focus>_<" + os.linesep
            elif annotation.get("headwearLikelihood") == "POSSIBLE" or annotation.get("headwearLikelihood") == "LIKELY" or annotation.get("headwearLikelihood") == "VERY_LIKELY":
                text = text + u"Are you wearing a hat?" + os.linesep
            else:
                text = text + u"usually?" + os.linesep
            count = count + 1
        text = text + os.linesep

    annotations = response['responses'][0].get('landmarkAnnotations')
    if annotations != None:
        text = text + u"Oh, maybe this is the place!" + os.linesep
        for annotation in annotations:
            text = text + u'[ ' + annotation.get("description") + u' ]' + os.linesep
        text = text + os.linesep

    annotations = response['responses'][0].get('logoAnnotations')
    if annotations != None:
        text = text + u"Oh, I know this logo." + os.linesep
        for annotation in annotations:
            text = text + u'[ ' + annotation.get("description") + u' ]' + os.linesep
        text = text + os.linesep

    print text
    return text


def get_vision_service():
    credentials = ServiceAccountCredentials.from_json_keyfile_dict(GOOGLE_APPLICATION_CREDENTIALS)
    return discovery.build('vision', 'v1', credentials=credentials,
                           discoveryServiceUrl=DISCOVERY_URL)

Credentials are obtained in ServiceAccountCredentials instead of GoogleCredentials in Sample. I didn't want to save it (I wanted to pass it via API Gateway). It's not implemented like line.py.

Impressions

I think that LINE BOT will deliver Slack-like convenience to many users, so I feel that there is great potential.

Even if it is only in-house, what you are doing in business, such as provisioning AWS resources, checking access status, etc. I feel like I can use it.

For the time being, let's put in an implementation that makes the message funny from the result obtained by LABEL_DETECTION of Vision API!

Recommended Posts

Python calling Google Cloud Vision API from LINE BOT via AWS Lambda
Use Google Cloud Vision API from Python
LINE BOT with Python + AWS Lambda + API Gateway
Google Cloud Vision API sample for python
Use Google Analytics API from Python
LINE BOT (Messaging API) development with API Gateway and Lambda (Python) [Part 2]
I tried to extract characters from subtitles (OpenCV: Google Cloud Vision API)
Detect Japanese characters from images using Google's Cloud Vision API in Python
Text extraction with GCP Cloud Vision API (Python3.6)
Try implementing a Cisco Spark bot with AWS Lambda + Amazon API Gateway (Python)
I tried using the Google Cloud Vision API
Amazon API Gateway and AWS Lambda Python version
[GCP] [Python] Deploy API serverless with Google Cloud Functions!
Make a parrot return LINE Bot on AWS Cloud9
From python to running instance on google cloud platform
[Python] Using Line API [1st Creation of Beauty Bot]
Procedure for creating a Line Bot on AWS Lambda
Speech transcription procedure using Python and Google Cloud Speech API
[LINE Messaging API] Create parrot return BOT with Python
Tweet from AWS Lambda
Made "Unofficial Apple Refurbished Product Introduction" BOT with LINE Messaging API (v2) + API Gateway + lambda (python)
AWS Amplify + API Gateway + Lambda + Python returns a normal response
Image collection by calling Bing Image Search API v5 from Python
Notes on accessing SQS from AWS VPC Lambda via endpoint
I want to send a message from Python to LINE Bot
Try to determine food photos using Google Cloud Vision API
Get data from analytics API with Google API Client for python
I tried the Google Cloud Vision API for the first time
Create API with Python, lambda, API Gateway quickly using AWS SAM
Python: Extract file information from shared drive with Google Drive API
[Python] Scraping in AWS Lambda
AWS CDK-Lambda + API Gateway (Python)
Query Athena from Lambda Python
Google Drive Api Tips (Python)
Use e-Stat API from Python
A story that I was addicted to calling Lambda from AWS Lambda.
Stream speech recognition using Google Cloud Speech gRPC API on python3 on Mac!
Until you try the Google Cloud Vision API (harmful image detection)
Things to note when running Python on EC2 from AWS Lambda
Manipulating kintone data with Python & C Data ODBC Driver from AWS Lambda
How to connect to Cloud Firestore from Google Cloud Functions with python code
[LINE Messaging API] Create a BOT that connects with someone with Python
Let's touch Google's Vision API from Python for the time being
[Python] I wrote a REST API using AWS API Gateway and Lambda.