[PYTHON] Using COTOHA, I tried to follow the emotional course of Run, Melos!

Tyrant Dionis: Yeah, what happened?

Meros: It's not happening!

I would like to read about "Run, Melos!" Using the sentiment analysis of COTOHA.

Process flow

1. Obtained the full text of "Run, Melos!" From Aozora Bunko
2. Run the full text of Hashire Melos to COTOHA (sentiment analysis).
3. Graph and analyze emotional values.

Advance preparation

Clone Aozora Bunko

!git clone --branch master --depth 1 https://github.com/aozorabunko/aozorabunko.git

code

from bs4 import BeautifulSoup

def get_word():

  #Specify the path from the cloned html(Run, Melos!)
  path_to_html='aozorabunko/cards/000035/files/1567_14913.html'
  
  #HTML parsing with BeautifulSoup
  with open(path_to_html, 'rb') as html:
    soup = BeautifulSoup(html, 'lxml')
  main_text = soup.find("div", class_='main_text')
  for yomigana in main_text.find_all(["rp","h4","rt"]):
    yomigana.decompose()
  sentences = [line.strip() for line in main_text.text.strip().splitlines()]
  aozora_text=','.join(sentences)

  aozora_text_list = aozora_text.split("。")
  return aozora_text_list

mport os
import urllib.request
import json
import configparser
import codecs
import sys
import time


client_id = "Your client ID"
client_secret = "Your own secret key"

developer_api_base_url = "https://api.ce-cotoha.com/api/dev/nlp/"
access_token_publish_url = "https://api.ce-cotoha.com/v1/oauth/accesstokens"

def cotoha_call(sentence_1,api_type):
    #Get access token
    def getAccessToken():     
        url = access_token_publish_url
        headers={
            "Content-Type": "application/json;charset=UTF-8"
        }
        data = {
            "grantType": "client_credentials",
            "clientId": client_id,
            "clientSecret": client_secret
        }
        data = json.dumps(data).encode()
        req = urllib.request.Request(url, data, headers)
        res = urllib.request.urlopen(req)
        res_body = res.read()
        res_body = json.loads(res_body)
        access_token = res_body["access_token"]
        return access_token

    #API URL specification(Named entity recognition)
    base_url_footer = "v1/"  + api_type
    url = developer_api_base_url + base_url_footer
    headers={
        "Authorization": "Bearer " + getAccessToken(), #access_token,
        "Content-Type": "application/json;charset=UTF-8",
    }
    data = {
        "sentence": sentence_1
    }
    data = json.dumps(data).encode()
    time.sleep(0.5)
    req = urllib.request.Request(url, data, headers)
        
    try:
        res = urllib.request.urlopen(req)
    #What to do if an error occurs in the request
    except urllib.request.HTTPError as e:
        #If the status code is 401 Unauthorized or 500 Internal Server Error, reacquire the access token and request again.
        if e.code == 401 or 500:
            access_token = getAccessToken()
            headers["Authorization"] = "Bearer " + access_token
            time.sleep(0.5)
            req = urllib.request.Request(url, data, headers)
            res = urllib.request.urlopen(req)
        #Show cause for errors other than 401 or 500
        else:
            print ("<Error> " + e.reason)
            #sys.exit()

    res_body = res.read()
    res_body = json.loads(res_body)
    return res_body


sentence = get_word()
end = len(a)
full_text = len(''.join(sentence))
sentiment_list = []
cnt = 0
wcnt = 0
for i in a:
  cnt+=1
  print(str(cnt)+ "/" + str(end))
  b = cotoha_call(i,"sentiment")
  sentiment_list.append([b['result']['sentiment'],b['result']['score'],i,wcnt])
  wcnt+=len(i)

#Library used for numerical calculation
import numpy as np
import pandas as pd
import seaborn as sns

#Library for drawing graphs
from matplotlib import pyplot as plt

#Specification for displaying the graph in colab
%matplotlib inline

#Adjust values for graphs
graph_x = []
graph_y = []
for i in sentiment_list:
  graph_x.append(i[3])
  if(i[0] == 'Negative'):
    tmp_y = (i[1] * -1) + (-1)
    graph_y.append(tmp_y)

  if(i[0] == 'Positive'):
    tmp_y = i[1]+1
    graph_y.append(tmp_y)
  
  if(i[0] == 'Neutral'):
    graph_y.append(i[1])

sns.set()

#plt.plot(x, y, color = 'black')
plt.figure(figsize=(40, 8), dpi=100)
plt.plot(graph_x, graph_y, color = 'red', marker = 'o', linestyle = '-', label = 'Sensor1')
#plt.xlim([0,5000])
plt.title("Run, Melos,")
plt.xlabel("Sentence")
plt.ylabel("Emotion")

Output result

Line graph

The vertical axis is emotion and the horizontal axis is the text position of each phrase. There is quite a range of emotional fluctuations.

Positive side analysis

Top positive phrases No1 ~ 3

[1.8126587191707609, 'I also bought a beautiful costume']
[1.6346889479386029, 'Your life is important now'],
[1.633215237292569, 'The young mason also screamed, running after Melos.']

The most positive phrase is ** "I bought a beautiful costume" **, with a big difference in scores between 2nd and 3rd place.

This is one phrase in the conversation between "Meros" and "Sister" when "Meros" returned to the village. I think I was very happy that I was able to run all the way from the village to the city and hand over the costumes I bought to my sister. (It's the original purpose of Meros in the first place.)

Sentences before and after the phrase that seems to be the most positive(reference)

I was surprised to find the exhausted appearance of my brother who staggered and walked. Then I asked my noisy brother a question.
"Nothing." Melos tried to force him to laugh. "I've left an errand in the city. I have to go to the city again soon. Tomorrow, I'll have your wedding. It's better to be early."
My sister raised her cheeks.
"I'm happy. I bought a beautiful costume. Now, let's go and let the villagers know. The wedding is tomorrow."
Melos also staggered, returned home to decorate the altar of the gods, set up a feast, and soon fell to the floor and fell into a deep sleep that could not breathe.

Negative side analysis

Top No. 1 to 3 of the most negative phrases

[-1.9459608426670426, 'Don't let the man die']
[-1.939408157054805, 'Even if it's a moment, it can't be wasted']
[-1.8666593927565555, 'Oh, the sun sets']

The 1st and 2nd places don't look like negative phrases, so they don't look right. ("Die", "Waste" This word seems to be the cause.)

So, isn't the practically most negative phrase ** "Oh, the sun sets" **?

It's a sentence while I'm returning to the city with all my strength by the time limit to release my friend Serinuntius who is hostage.

Multiple difficulties stand in front of Meros, but we will break through them Even though I finally kneel once, various thoughts go around and in the depiction of standing up again It is a depiction of the part that time is passing every moment while doing this.

Sentences before and after the phrase that seems to be the most negative(reference)

Run! Meros.
I am trusted. I am trusted. That devil's whisper earlier was a dream. It's a bad dream. Forget it.
When the five organs are tired, they have such a bad dream. Meros, it's not your shame.
After all, you are a true hero. Isn't it possible to stand and run again? Thank you!
I can die as a justice fighter. Oh, the sun goes down. It sinks steadily. Wait, Zeus.
I have been an honest man since I was born. Leave it as an honest man and let him die.

If you analyze the emotional value in blocks ...

The negative and positive emotional values, the parts where each is organized, are shown below in blocks.

• Change the graph description part described above to the following scatter plot and execute.

Scatter plot display

#plt.plot(graph_x, graph_y, color = 'red', marker = 'o', linestyle = '-', label = 'Sensor1')
plt.scatter(graph_x, graph_y, color = 'red', marker = 'o', linestyle = '-', label = 'Sensor1')

Scatter plot

Blocks with the most positive values in a row (the part marked with an orange circle)

python

Hope to fulfill the obligation.
Hope to kill myself and protect my honor.
The sunshine casts red light on the leaves of the trees, and the leaves and branches are shining as they burn.
There is still time before sunset. Someone is waiting for me.
There are people who are quietly expecting me without any doubt.
I am believed. My mystery of life doesn't matter. I can't say anything nice about being dead and apologizing.
I have to reward my trust. That's just one thing now.
Run! Meros.
I am trusted. I am trusted. That devil's whisper earlier was a dream. It's a bad dream. Forget it.
When the five organs are tired, they have such a bad dream. Meros, it's not your shame.
After all, you are a true hero. Isn't it possible to stand and run again?
Thank you! I can die as a justice fighter.

It's a depiction of Melos, who once knelt down, when he stands up again. You can read the heightened emotions. (By the way, it is the sentence immediately before the most negative phrase described above.)

Blocks with consecutive most negative values (marked in blue)

python

If that happens, I'll be harder than dying. I'm a traitor forever. The most disgraceful race on earth.
Serinuntius, I'll die too. Let me die with you. Only you must believe me.
No, is that also my selfishness? Ah, let's survive as a villain.
My house is in the village. There are also sheep. My sister and her husband would never drive me out of the village.

It's a depiction when trying to give up helping a friend Seriuntius. Certainly, it's thrown away. (After that, I throw out my limbs, and when I finally get sick, I get drowsy.)

Summary

~~ From the positive emotional value, I thought that Melos was a sister complex. ~~

It's easy, but I tried to understand "Run, Melos!" Using sentiment analysis. It's quite interesting to read the sentences like this.

that's all. Thank you for reading this far.