[PYTHON] I tried to erase the negative part of Meros

Introduction

I touched COTOHA API the other day, so here is what I did in that flow. Please see Article posted the other day for the COTOHA API and its introduction.

What i did

Use the COTOHA API to purge the negative parts of Meros.

  1. Clone Aozora Bunko's git and run to find Melos!

Very heavy


$ git clone [email protected]:aozorabunko/aozorabunko.git

--Copy the text of Run Meros to a text file

  1. Shape the text of Run Meros

--Split sentences by punctuation --Remove unnecessary symbols such as ""

  1. Perform sentiment analysis for each sentence and remove those that are judged negative
The code is long so it's collapsed
import requests
import json
import sys
import time


BASE_URL = "https://api.ce-cotoha.com/api/dev/nlp/"
CLIENT_ID = ""  #Enter the issued ID
CLIENT_SECRET = ""  #Enter the issued secret


#Get an access token
def auth(client_id, client_secret):
    token_url = "https://api.ce-cotoha.com/v1/oauth/accesstokens"
    headers = {
        "Content-Type": "application/json",
        "charset": "UTF-8"
    }
    data = {
        "grantType": "client_credentials",
        "clientId": client_id,
        "clientSecret": client_secret
    }
    r = requests.post(token_url,
                      headers=headers,
                      data=json.dumps(data))
    return r.json()["access_token"]


#Call sentiment analysis api
def sentiment(sentence, access_token):
    base_url = BASE_URL
    headers = {
        "Content-Type": "application/json",
        "charset": "UTF-8",
        "Authorization": "Bearer {}".format(access_token)
    }
    data = {
        "sentence": sentence
    }
    r = requests.post(base_url + "v1/sentiment",
                      headers=headers,
                      data=json.dumps(data))
    return r.json()

if __name__ == "__main__":
    document = "Meros was furious."
    args = sys.argv
    if len(args) >= 2:
        document = str(args[1])

    access_token = auth(CLIENT_ID, CLIENT_SECRET)


    #Formatting into parseable sentences(Separation by punctuation, symbol removal, etc.)
    sentences = splitwords(document)

    f = open('opt.txt', 'w')
    for sentence in sentences:
        if not sentence:
            continue

        res = sentiment(sentence, access_token)
        print(res)

        if res['result']['sentiment'] == 'Negative':
            continue

        f.write(sentence + "\n")
    f.close()

    print("deleted negative melos")

result

Here are some sentences from the beginning.

Meros


Meros was furious
I decided that I had to get rid of the king of evil wisdom and violence.
Meros doesn't understand politics
Meros is a village shepherd
I blew a whistle and lived playing with sheep

Melos not negative


I decided that I had to get rid of the king of evil wisdom and violence.
Meros is a village shepherd
I blew a whistle and lived playing with sheep

Meros is no longer angry and now understands politics.

Execution result


$ python melos.py melos.txt

{'result': {'sentiment': 'Negative', 'score': 0.7040895800341446, 'emotional_phrase': [{'form': 'Furious', 'emotion': 'N'}]}, 'status': 0, 'message': 'OK'}
{'result': {'sentiment': 'Neutral', 'score': 0.34207414196325664, 'emotional_phrase': []}, 'status': 0, 'message': 'OK'}
{'result': {'sentiment': 'Negative', 'score': 0.33534926492915057, 'emotional_phrase': [{'form': 'I don't know', 'emotion': 'N'}]}, 'status': 0, 'message': 'OK'}
{'result': {'sentiment': 'Neutral', 'score': 0.4202261100237903, 'emotional_phrase': []}, 'status': 0, 'message': 'OK'}
{'result': {'sentiment': 'Neutral', 'score': 0.38977432300637777, 'emotional_phrase': []}, 'status': 0, 'message': 'OK'}
...

Looking at the execution result, "furious" and "don't know" are Negative judgments.

Comparing the results of the full text, the original is ** 464 lines **, but after the purge it is ** 357 lines **, so ** 107 lines ** have disappeared.

Full text of Meros after the purge
I decided that I had to get rid of the king of evil wisdom and violence.
Meros is a village shepherd
I blew a whistle and lived playing with sheep
Today early dawn Meros left the village, crossed the fields, crossed the mountains, and came to this city of Syracuse, which was 10 ri away.
Meros has neither a father nor a mother
There is no wife
I live with my 16 shy sisters
This younger sister was to welcome a savvy shepherd from the village as a groom in the near future.
The wedding is just around the corner
Melos, therefore, came all the way to the city to buy brides' costumes and feast treats.
First, I bought the items and then wandered around the main road of the city.
Meros had a friend of stilts
Serinuntius
Now I'm a mason in this city of Syracuse
I'm going to visit that friend from now on
I haven't seen him in a long time, so I'm looking forward to visiting him.
While walking, Melos was suspicious of the city.
Quiet
I caught the young people I met on the road and asked if something had happened, and when I came to this city two years ago, everyone sang even at night and the city should have been lively.
The young people shook their heads and did not answer
I walked for a while and met an old man, and this time I asked him more strongly.
The old man did not answer
Melos shook the old man's body with both hands and asked questions.
The king kills a person
Why kill
It is said that he has nausea, but no one has such nausea.
Did you kill a lot of people?
Yes, at first the King's sister-in-law
Then, your own heritage
Then my sister
Then, my sister's child
Then the Empress
Then, the wise servant Alekis
Surprised
Is the King mad?
No, I'm not confused
You can't believe in people.
These days, even the hearts of his vassals are suspicious, and he is ordering those who are living a little flashy to offer hostages one by one.
Six people were killed today
Melos was a simple man
I went shopping to the royal castle with my back on my back
Immediately he was arrested by a patrol policeman
After being investigated, a dagger came out of Melos's pocket, so the noise became louder.
Meros was pulled out before the king
What were you going to do with this dagger
say! Tyrant Dionis asked quietly, but with dignity
The king's face was pale and the wrinkles between his eyebrows were as deep as engraved.
Save the city from the hands of tyrants
Are you? The king laughed
It can't be helped
Do not say! Melos stood up and refuted
You are the ones who told me that it was a legitimate attitude to doubt.
Human beings are originally a mass of selfishness
Believe it or not
The tyrant calmed down and muttered, sighing relievedly
I want peace
What is peace for
Is it to protect my position?
What is peace, killing innocent people
A person who is fooled
The king quickly raised his face and rewarded him.
With your mouth, you can say anything pure
I can't see the depths of a person's belly cotton
You don't even hear that you cried and apologized since you became a crucifixion.
Oh, the king is clever
I'm prepared to die
I want my only sister to have a host
Within three days I will have a wedding in the village and I will definitely come back here
stupid
Laughed low with a hoarse voice
I'll tell a ridiculous lie
Is the missed bird coming back?
That's right
Will come back
I keep my promise
Please forgive me for only three days
My sister is waiting for my return
If you can't believe me so much, okay, there is a mason named Serinuntius in this city
My one and only friend
Let's leave that as a hostage here
If I run away and don't come back here until Higurashi on the third day, strangle that friend
Enjoy, please do so
Upon hearing that, the king softly laughed at the north with a cruel feeling.
I have decided not to come back anyway
It's also nice to kill a substitute man on the third day.
I want to show off to the honest people in the world.
I heard your wish
You should call that substitute
Come back by sunset on the third day
If you give me that, I'm sure I'll kill you
I hope you come a little late
I'll forgive your sins forever
What are you saying
my mother
If life is important, come late
I know your heart
I don't want to say anything
Stilts friend Serinuntius was called to the royal castle at midnight
In front of the tyrant Dionis, a good friend and a good friend meet for the first time in two years
Melos told his friend everything
Serinuntius silently nodded and hugged Melos.
That was good between friends
Serinuntius was struck by a rope
Meros left immediately
Early summer, a star full of sky
Melos rushed down the road of Juri without a sleep that night, and arrived at the village the next morning, when the sun had already risen high and the villagers had gone out into the field to start work.
Melos's sixteen sisters were also in charge of the flock today instead of their brothers.
I was surprised to find the exhausted appearance of my brother who staggered and walked
nothing
Meros tried to laugh forcibly
I left an errand in the city
I have to go to the city again soon
Tomorrow, give your wedding
It ’s better to be early
Are you happy
I also bought a beautiful costume
Come on, let the villagers know
The wedding is tomorrow
Melos also staggered, returned home to decorate the altar of the gods, set up a feast, and soon fell to the floor and fell into a deep sleep that could not breathe.
I woke up at night
Melos visited the groom's house as soon as he got up
Then, because of some circumstances, I asked him to have a wedding tomorrow.
Meros pushed me further, saying I can't wait, please give me tomorrow.
The son-in-law's shepherd was also strong
The wedding was held at noon
When the bride and groom were sworn in to the gods, black clouds covered the sky, and it began to rain, and eventually it became a heavy rain that swept the axles.
Meros was also full of joy and for a while even forgot that promise with the king.
The feast became turbulent and gorgeous at night, and people no longer care about heavy rain outside.
Meros wanted to stay here for the rest of his life
I wanted to live with these good people for the rest of my life, but now it's my body, not my own
It is something that cannot be left
There is still enough time before sunset tomorrow
I thought I'd take a rest and then start right away
By that time, it will be raining lightly
Tonight, I approached the bride who seems to be drunk with joy,
Congrats
As soon as you wake up, go to the city
I have an important business
Even without me, you'll never be lonely because you already have a kind host
You know that too
Don't make any secrets with your host
That's all I want to say to you
Your brother is probably a great man, so take pride in that too
The bride nodded with a dreamy feeling
Melos then tapped the groom's shoulder,
It is mutuality that there is no preparation
In my house, the only treasures are my sister and sheep.
Nothing else
I'll give you all
Another proud thing about becoming Melos' younger brother
Melos laughed and bowed to the villagers, walked away from the banquet, sneaked into the sheep shed, and slept deeply as if dead.
I woke up in the twilight the next day.
Meros jumped up, Minami Muzo, overslept, no, it's still okay, if you start soon, you'll be in time for the promised time
By all means, let's show that king where the faith of human beings exists today.
Then laugh and go up to the crucifixion stand
Meros began to dress calmly
It seems that it is raining a little.
I was ready
By the way, Melos shook his arms and ran like an arrow in the rain.
I will be killed tonight
Run to be killed
Run to save your substitute friend
Run to defeat the king's wisdom
I have to run
Then I will be killed
Protect your honor from a young age
Farewell, hometown
I almost stopped several times
By the time I left the village, crossed the fields, passed through the forest, and arrived at the neighboring village, the rain stopped, the sun rose high, and it was getting hot.
My sisters will surely be a good couple
I shouldn't have any concerns right now
If you go straight to the royal castle, that's fine.
You don't have to hurry so much
I regained my sullenness to walk slowly, and started singing my favorite small song with a good voice.
I walked around to Nisato and Sanri, and when I reached the middle of the whole ri, the calamity that had fallen, Meros's legs stopped.
Behold, the river ahead
The water source of the mountain was flooded by the heavy rain yesterday, and the muddy stream gathered downstream, destroying the bridge at once, and the torrent that echoed somehow bounced off the bridge girder to the fine dust of Konoha.
The flow is finally swelling and becoming like the sea
Oh, calm down, the raging flow! Time goes by every moment
The sun is already midday
If I couldn't get to the royal castle before it sank, that good friend would die for me.
Now I'm prepared for Melos
There is no other than swimming
Oh, see the gods too! Now is the time to show the great power of love and sincerity that is not defeated by muddy currents
While being swept away, I was able to cling to the trunk of the tree on the opposite bank.
Thank you
Meros made one big horse-like tremor and immediately hurried again.
The sun is already leaning west
wait
What to do
I have to go to the royal castle before the sun sets
Let go
Don't let go
Leave all your belongings
I have nothing but life
I'll give that one life to the king from now on.
I want that life
Well, at the command of the king, I was ambushing here.
The bandits swung the club all at once without saying anything.
I can't stand up
Ah, ah, swim through the muddy stream, shoot down three bandits, Skanda, Melos who has broken through so far
True hero, Melos
My beloved friend has to be killed just because he believed in you
I lay down on the grassland by the roadside
If you get tired physically, you can do your spirit together
I made so much effort
I didn't have the heart to break my promise
God also sees, I have tried my best
I ran until I couldn't move
Oh, if I can, I want to break my chest and see the crimson heart
I want to show you this heart that is moving only with the blood of love and faith
But I've run out of spirit and roots at this important time.
I'm sure you will be laughed at
My family is also laughed at
Falling down in the middle is the same as doing nothing from the beginning
This may be my destiny
Serinuntius, forgive me
You always believed in me
I didn't deceive you either
We were really good friends and friends
Still, you will be waiting for me innocently
Oh i'll be waiting
Thank you, Serinuntius
Well believed me
The faith between friends is the most proud treasure in the world.
Serinuntius, I ran
Believe me! I came here in a hurry
Break through the muddy stream
Even from the bandit's enclosure, he slipped through and came down the pass at once.
Because I was able to do it
Oh, on top of that, don't want me
Leave it alone
I don't care
Untidy
Laugh
The king told me to come a little late
If you give me, I promised to kill you and help me
But now, I'm at the mercy of the king
I will be late
The king will laugh at me alone and release me without doing so
I'm a traitor forever
Serinuntius, I'll die too
Only you must believe me
My house is in the village
There are also sheep
Justice, faith, love, if you think about it, it's crap
Kill people and live yourself
Wasn't it the law of the human world?
Yan Nuruya
――I threw my limbs out, and at last I was drowsy
Suddenly, I could hear the sound of flowing water in my ears.
It seems that water is flowing right at your feet
When I stood up and looked at it, fresh water was gushing out from the crevice of the rock, whispering something small.
Melos crouched to be sucked into the fountain
I scooped water with both hands and drank a sip
Can walk
let's go
A little hope was born with the recovery of physical fatigue
Hope to fulfill the obligation
Hope to kill myself and protect my honor
There is still time before sunset
Someone is waiting for me
There are people who are quietly expecting me without any doubt.
I am believed
My life mystery doesn't matter
I can't say good things like apologize for being dead
Now that's just one thing
Run! Meros
I'm trusted
I'm trusted
That devil's whisper, that was a dream
Forget
Meros, not your shame
After all, you are a true hero
Isn't it possible to stand and run again?
Thank you! I can die as a justice fighter
Wait, Zeus
I have been an honest man since I was born
By now, that man is also on the crucifixion
Oh, that guy, for that guy I'm running so much now
Hurry up, Melos
Don't be late
Now is the time to let us know the power of love and sincerity
Meros was almost naked now
I couldn't breathe, and blood spewed out of my mouth twice or three times.
appear
Far away, you can see the tower of the city of Syracuse
The tower is shining in the setting sun
Oh, Melos
Who
Melos asked while running
This is Philostratus
I am a disciple of your friend Serinuntius
The young mason also screamed as he ran after Melos.
It's no good anymore
Please stop running
I can't help him anymore
No, the sun hasn't set yet
Right now, that person is about to be sentenced to death
Oh you were late
If it was a little bit faster!
No, the sun hasn't set yet
There is nothing more than running
Please stop
Please stop running
Your life is important now
He believed in you
Even if I was pulled out to the prison, I was fine
That's why I run
Run because you are believed
It doesn't matter if you're in time or not
Human life doesn't matter either
Follow me! Philostratus
Oh, did you go crazy
Then you should run
Maybe it's not too late
Good to run
To say
The sun hasn't set yet
With the last force of mortality, Melos ran
Meros's head is empty
It was in time
wait
Don't kill that person
Meros is back
As promised, I'm back now
The crucifixion pillars have already been erected and the rope has been struck, and Serinuntius is gradually caught.
Meros witnessed it and squeezed the crowd as if he had swam in the last brave, predecessor, muddy stream,
I'm a prisoner! I am the one who will be killed
It's Meros
I'm here to take him hostage! While screaming with a faint voice, he finally climbed up to the crucifixion and clung to both feet of his friend who was being caught.
The crowd screamed
Appreciate
Forgive me
The rope of Serinuntius was unwound.
Serinuntius
Hit me
Hit your cheeks all at once
If you didn't hit me, I wouldn't even be eligible to hug you
Beat
Serinuntius nodded as if he had seen everything, and hit Melos' right cheek so loudly that it rang in the prison.
After hitting, smile gently,
Meros, hit me
Hit my cheek as loudly
Melos snarled in his arm and hit Serinuntius' cheek
Thank you, friend
I could hear the voice of mourning from the crowd
Your wish has come true
You have won my heart
Would you please join me as well?
Please listen to my wishes and make me one of your companions.
Cheers broke out among the crowd
Hurray, hurray the king
A girl dedicated a scarlet cloak to Melos
A good friend taught me with care
Meros, aren't you a bluff?
You should wear the cloak as soon as possible

Articles that I used as a reference

Ore Program Ugokas Omae Genshijin Naru Using COTOHA, I tried to follow the emotional course of Run, Melos!

in conclusion

~~ I read Melos for the first time in a long time, but it was very easy to read and interesting. ~~ I wanted to do something using Python and COTOHA, but I was able to create it quickly because it almost just calls the API. I also read Meros after the purge, but I felt that the whole story was organized enough to be grasped. (This may be on the side of Melos' style) In addition to sentiment analysis, COTOHA has an API that seems to be interesting, so I would like to play with it as well.

Recommended Posts

I tried to erase the negative part of Meros
I tried to touch the API of ebay
I tried to correct the keystone of the image
I tried to predict the price of ETF
I tried to vectorize the lyrics of Hinatazaka46!
I tried to summarize the basic form of GPLVM
I tried to visualize the spacha information of VTuber
I tried to classify the voices of voice actors
I tried to summarize the string operations of Python
I tried to move the ball
I tried to estimate the interval.
I tried to find the entropy of the image with python
I tried to get the location information of Odakyu Bus
I tried to find the average of the sequence with TensorFlow
[Python] I tried to visualize the follow relationship of Twitter
[Machine learning] I tried to summarize the theory of Adaboost
I tried to fight the Local Minimum of Goldstein-Price Function
[Linux] I tried to summarize the command of resource confirmation system
I tried the asynchronous server of Django 3.0
I tried to summarize the umask command
I tried to get the index of the list using the enumerate function
I tried to automate the watering of the planter with Raspberry Pi
I want to handle the rhyme part1
I tried to recognize the wake word
I want to handle the rhyme part3
I tried to summarize the graphical modeling.
I tried to estimate the pi stochastically
I tried to touch the COTOHA API
I tried to expand the size of the logical volume with LVM
I tried running the DNN part of OpenPose with Chainer CPU
I tried to summarize the frequently used implementation method of pytest-mock
I tried to improve the efficiency of daily work with Python
I tried to visualize the common condition of VTuber channel viewers
I want to handle the rhyme part2
I want to handle the rhyme part5
I want to handle the rhyme part4
I tried to transform the face image using sparse_image_warp of TensorFlow Addons
I tried to visualize the age group and rate distribution of Atcoder
I tried transcribing the news of the example business integration to Amazon Transcribe
zoom I tried to quantify the degree of excitement of the story at the meeting
I tried to estimate the similarity of the question intent using gensim's Doc2Vec
I tried how to improve the accuracy of my own Neural Network
I tried to solve the 2020 version of 100 language processing [Chapter 3: Regular expressions 25-29]
I tried to get the authentication code of Qiita API with Python.
I tried to automatically extract the movements of PES players with software
(Python) I tried to analyze 1 million hands ~ I tried to estimate the number of AA ~
I tried to summarize the logical way of thinking about object orientation.
I tried to find the optimal path of the dreamland by (quantum) annealing
I tried to extract and illustrate the stage of the story using COTOHA
I tried to verify and analyze the acceleration of Python by Cython
I tried to analyze the negativeness of Nono Morikubo. [Compare with Posipa]
I tried to streamline the standard role of new employees with Python
I tried to visualize the text of the novel "Weathering with You" with WordCloud
[Linux] I tried to verify the secure confirmation method of FQDN (CentOS7)
I tried to get the RSS of the top song of the iTunes store automatically
I tried to get the movie information of TMDb API with Python
I tried to display the altitude value of DTM in a graph
I tried the common story of using Deep Learning to predict the Nikkei 225
Using COTOHA, I tried to follow the emotional course of Run, Melos!
I tried to verify the result of A / B test by chi-square test
I tried to predict the behavior of the new coronavirus with the SEIR model.