[PYTHON] A story about trying to reproduce Katsuwo Isono, who does not react to inconvenience, by natural language processing.

Introduction

COTOHA API is a Japanese natural language processing API. It has various functions and can be used for 1000 / day even with a free account. It is a very fun API.

COTOHA API

By the way, do you all know Isono-kun? Yes, it's Katsuo Isono, a national anime.

This Isono-kun is called something in the play, but when it is inconvenient, he does not hear it or reacts so-called "human". This time, I will try various things with COTOHA API to realize this.

For details on how to use the COTOHA API, please refer to the [article] I wrote earlier (https://qiita.com/mosamosa/items/e63f6e582a206659dc2b) and various other people who have written articles. ..

The completed ISONO reacts like this.

I:Hey Isono. Let's play baseball.
Isono:Read?

I:Isono-kun
Isono:Read?

I:Isono is no good
Isono:...

I:Isono-kun is cool
Isono:Read?

I:Speaking of Isono, Iso Nori
Isono:...

The number of samples is small, but what about? Isn't it like Isono?

Preparation

In the COTOHA API, create a wrapper function to display whether it recognizes that "Isono was called". If you think you have been called, "Did you call?" If not, "..." is displayed.

Folding because it's not important 
cotoha_exp = [
    "Isono",
    "Isono-kun",
]
def call_cotoha(res_func,texts,cotoha_token,cotoha_exp):
    """
    res_func:A function that determines if it has been called. Receives a statement and returns true false.
    texts:Sentence array to be judged
    cotoha_token:Access token
    cotoha_exp:For determining Isono and its name.
    """
    for t in texts:
        does_ans = res_func(t,cotoha_token,cotoha_exp)
        if does_ans:
            answer = "Read?"
        else:
            answer = "..."
        print("I:{} \n Isono:{}\n".format(t,answer))

1. The simplest way to call (morphological analysis, isono-kun with excessive self-consciousness)

The simplest way for Isono-kun to detect his call is to morphologically analyze the text and react if it contains Isono or something similar.

Simply put, if Isono is included in the sentence, it will react.

The parsing documentation is here [https://api.ce-cotoha.com/contents/reference/apireference.html#parsing_response_morpheme)

Source code. Fold it because the article will be long 
def parser_base(text,cotoha_token,cotoha_exp):

    header = {
        "Content-Type":"application/json",
        "Authorization":"Bearer "+cotoha_token
    }

    datas = {
        "sentence":text
    }

    r = requests.post(api_base+"nlp/v1/parse",headers=header,data=json.dumps(datas))
    parsed = json.loads(r.text)
    for res in parsed["result"]:
        for tok in res["tokens"]:
            for exp in cotoha_exp:
                if exp in tok["form"]:
                    return True
    return False

result

call_cotoha(parser_base,texts,cotoha_token,cotoha_exp)

I:Hey Isono. Let's play baseball.
Isono:Read?

I:Isono-kun
Isono:Read?

I:Isono is no good
Isono:Read?

I:Isono-kun is cool
Isono:Read?

I:Speaking of Isono, Iso Nori
Isono:Read?

Of course, it's natural, but it reacts to everything. I feel a little over-conscious.

2. Reaction if it is included in the named entity (after all, excessive self-consciousness)

Even if you don't bother to morphologically analyze and see all the tokens, the COTOHA API has a function called named entity recognition, which makes it easy to extract personal names, place names, numerical expressions, etc. If you use this, you can do something similar to 1. in a slightly stylish way.

Source code. Fold it because the article will be long 
def ne_base(text,cotoha_token,cotoha_exp):

    header = {
        "Content-Type":"application/json",
        "Authorization":"Bearer "+cotoha_token
    }

    datas = {
        "sentence":text
    }

    r = requests.post(api_base+"nlp/v1/ne",headers=header,data=json.dumps(datas))
    parsed = json.loads(r.text)
    for res in parsed["result"]:
        for exp in cotoha_exp:
            if exp in res["form"]:
                return True
    return False

result

I:Hey Isono. Let's play baseball.
Isono:Read?

I:Isono-kun
Isono:Read?

I:Isono is no good
Isono:Read?

I:Isono-kun is cool
Isono:Read?

I:Speaking of Isono, Iso Nori
Isono:Read?

After all it is excessive self-consciousness. Well, it's about age. The reputation of the people around me is anxious.

3. Reaction if included in the keyword (Isono-kun who does not react at all)

Unlike named entity extraction, the COTOHA API has a function called keyword extraction, and I would like to make Isono react when it becomes a keyword in a conversation. If you feel that you are the subject, you should react.

Source code. Fold it because the article will be long 
def key_base(text,cotoha_token,cotoha_exp):

    header = {
        "Content-Type":"application/json",
        "Authorization":"Bearer "+cotoha_token
    }

    datas = {
        "document":text
    }

    r = requests.post(api_base+"nlp/v1/keyword",headers=header,data=json.dumps(datas))
    parsed = json.loads(r.text)
    for res in parsed["result"]:
        for exp in cotoha_exp:
            if exp in res["form"]:
                return True
    return False

result

I:Hey Isono. Let's play baseball.
Isono:...

I:Isono-kun
Isono:...

I:Isono is no good
Isono:...

I:Isono-kun is cool
Isono:...

I:Speaking of Isono, Iso Nori
Isono:...

It doesn't react at all. Personal names do not seem to be keywords. This is Isono-kun's reaction when he did something wrong.

Will it respond if I call it even with this?

I:Isono-kun, Isono-kun, Isono-kun, Isono-kun, Isono-kun
Isono:...

It's no good ... Isono-kun seems to be dead ...

4. Use similarity (good feeling)

The COTOHA API has a function to judge the similarity between two sentences, so I would like to use this. Specifically, if the similarity with the sentence "Call Isono" is high, I will judge that Isono is being called.

Source code. Fold it because the article will be long 
def sim_base(text,cotoha_token,cotoha_exp):

    header = {
        "Content-Type":"application/json",
        "Authorization":"Bearer "+cotoha_token
    }

    datas = {
        "s1":text,
        "s2":"Call Isono"
    }

    r = requests.post(api_base+"nlp/v1/similarity",headers=header,data=json.dumps(datas))
    parsed = json.loads(r.text)
    if parsed["result"]["score"] > 0.5:
        return True
    else:
        return False

result

I:Hey Isono. Let's play baseball.
Isono:Read?

I:Isono-kun
Isono:Read?

I:Isono is no good
Isono:...

I:Isono-kun is cool
Isono:...

I:Speaking of Isono, Iso Nori
Isono:...

Oh, isn't it pretty good? Only the first and second call sentences responded. There is no reaction method that is quite practical for wakeups such as chatbots.

5. React only when the sentence type is interrogative (subtle ...)

The COTOHA API can determine what type of statement a sentence is. (Descriptions / questions / instructions can be taken in more detail. For details, see Documentation.)

I will try to use this to react only when a sentence is interrogative. (I wish I had a sentence type called a call, but I couldn't find it ...)

Of course, if it is an interrogative form, it will not work, so if it is an interrogative form, morphological analysis, which is considered to be the most sensitive, is performed to determine whether COTOHA is included.

Source code. Fold it because the article will be long 
def parser_sent_type_base(text,cotoha_token,cotoha_exp):
    header = {
        "Content-Type":"application/json",
        "Authorization":"Bearer "+cotoha_token
    }
    datas = {
        "sentence":text
    }

    r = requests.post(api_base+"nlp/v1/sentence_type",headers=header,data=json.dumps(datas))
    parsed = json.loads(r.text)
    if parsed["result"]["modality"] == "interrogative":
        return parser_base(text,cotoha_token,cotoha_exp)
    else:
        return False

result

I:Hey Isono. Let's play baseball.
Isono:...

I:Isono-kun
Isono:...

I:Isono is no good
Isono:...

I:Isono-kun is cool
Isono:...

I:Speaking of Isono, Iso Nori
Isono:...

Hmm ... subtle ... it looks like no good.

6. React only when you are likely to be complimented (convenient Isono)

The COTOHA API has a function called sentiment analysis that allows you to determine whether a sentence is positive or negative. So ** When the sentence is positive and contains Isono → When Isono is likely to be praised **

I just want to make it react. Isono-kun is convenient. Let's all get in the mood for Isono.

Source code. Fold it because the article will be long 
def senti_parser_base(text,cotoha_token,cotoha_exp):
    header = {
        "Content-Type":"application/json",
        "Authorization":"Bearer "+cotoha_token
    }
    datas = {
        "sentence":text
    }

    r = requests.post(api_base+"nlp/v1/sentiment",headers=header,data=json.dumps(datas))
    parsed = json.loads(r.text)
    if parsed["result"]["sentiment"] == "Positive":
        return parser_base(text,cotoha_token,cotoha_exp)
    else:
        return False

result

I:Hey Isono. Let's play baseball.
Isono:...

I:Isono-kun
Isono:...

I:Isono is no good
Isono:...

I:Isono-kun is cool
Isono:Read?

I:Speaking of Isono, Iso Nori
Isono:...

It seems that the characteristics of Isono can be reproduced quite well.

Isono, which I thought of.

From the results of the experiments so far

I feel that Isono can be reproduced by making it react when either of them is satisfied. I will try it.

def ISONO(text,cotoha_token,cotoha_exp):
    flag1 = sim_base(text,cotoha_token,cotoha_exp)
    flag2 = senti_parser_base(text,cotoha_token,cotoha_exp)
    return (flag1 or flag2)

result

I:Hey Isono. Let's play baseball.
Isono:Read?

I:Isono-kun
Isono:Read?

I:Isono is no good
Isono:...

I:Isono-kun is cool
Isono:Read?

I:Speaking of Isono, Iso Nori
Isono:...

Artificial intelligence ** ISONO ** Bomb here (exaggeration)

At the end.

I've tried various things using as many APIs as possible, but if you want to try some features that aren't available yet (summary, anaphora resolution, etc.), please register here.

COTOHA API Portal

I intended to write it completely, but I thought that it could be used for wakeups such as chatbots.

By the way, I feel that I can create a negative personality if I try to react only to sentences that are likely to be denied. I feel like I can reproduce it if I have a certain personality ...

** Swastika application swastika ** seems to be worthwhile.

Recommended Posts

A story about trying to reproduce Katsuwo Isono, who does not react to inconvenience, by natural language processing.
A story about a Python beginner who was about to be crushed by ModuleNotFoundError: No module named'tweepy'
[Python] Try to classify ramen shops by natural language processing
A story about trying to implement a private variable in Python.
A story about a student who does not know the machine learning machine learned machine learning (deep learning) for half a year
A story about trying to automate a chot when cooking for yourself
A story about trying to run multiple python versions (Mac edition)
Loose articles for those who want to start natural language processing
A story about trying to improve the testing process of a system written in C language for 20 years
Preparing to start natural language processing
A story about trying to run JavaScripthon on Windows and giving up.
A story about trying to connect to MySQL using Heroku and giving up
A story about a beginner trying hard to set up CentOS 8 (procedure memo)
3. Natural language processing with Python 1-2. How to create a corpus: Aozora Bunko