[PYTHON] Mitsuhiko "I can tell the criminal by introducing myself!"

prologue

Hello everyone. Today, I would like to introduce an epoch-making invention that helps me to solve the case. It's a system that collects the conversations of suspects before the incident and predicts the criminal and the victim! I'm still not sure about AI, but there's nothing impossible for the Boy Detectives!

Purpose

• Predict criminals and victims before an incident occurs

conditions

• The target case is a murder case
• Predict the criminal and victim from the conversation from the beginning of the story to before the incident

System overview

• Language is Python
• The library used for machine learning is keras
• The morphological analysis engine is janome

Implementation flow

I will briefly explain the procedure for creating a system.

• Data collection We collect data from Conan's books. It was difficult to pick up all the conversations until the incident, so I decided to use 10 balloons as learning data. In addition, all the person names have been converted to "Tanaka" so that the person names appearing in the conversation do not affect the judgment.

• Morphological analysis The collected data will be written in Japanese, but it cannot be used as learning data as it is, so we will process the data. By separating sentences into words, it is possible to perform vectorization that will appear later. For example, "Sumomomo Momomo no Uchi" is separated by "Sumomo / Momo / Momo / Momo / Momo / no / Uchi".

• Vectorization Convert sentences to vector data using doc2vec of janome. By expressing sentence data numerically, it can now be used as learning data.

• Learning Label the completed vector data as "criminal / victim / general public" and learn.

result

The accuracy was close to 100% for the training data, but the accuracy for the new data was almost nonexistent.

• Cause of defeat: The learning data was overwhelmingly insufficient (28)
• It was harder than I expected to manually convert the contents of the book into data.
• I compromised because the posting date was short
• Priority for reading manga

As is often said, the hardest part of machine learning is collecting learning data. I want to do something like this! I think it is rare to get data immediately when I think about it, so I thought it was important to keep an antenna on the data on a regular basis.

bonus

It is the result of judging the lines of various characters. From left to right, it is the probability of [general victim victim criminal].

• I am a cat
• [0.506109 0.47779834 0.01609264]
• Oss! Ora Goku
• [0.7559725 0.22846058 0.01556693]
• I Doraemon
• [0.6403506 0.3441662 0.01548317]
• Haruhi Suzumiya from Tochu. I'm not interested in just humans. If there are aliens, future people, different worlds, or psychics in this, come to me. that's all.
• [0.2473547 0.69323796 0.05940729]
• I will be the god of the new world!
• [0.28107646 0.70111287 0.01781074]
• If you give up, the match will end there
• [0.63113874 0.34889922 0.01996207]
• No results !! I didn't get any results !!
• [0.46731383 0.48855832 0.04412784]
• Seeing only one truth, it looks like a child, the brain is an adult, and its name is Detective Conan!
• [0.50221133 0.47582945 0.02195918]

It almost depends on the probability of the general public or the victim. The probability of a criminal is as low as 6% at the highest, and maybe you are learning that flashy lines are not the criminal.

The highest probability of being a victim was Kira's "I will be the god of the new world!"

Conclusion

Light Yagami dies