[PYTHON] [Note] WordCloud from morphological analysis

Move the text that was in the python file to another file

textmining_source.txt


What is scaffold?
A convenient tool that automatically sets all the basic actions in addition to creating the routes, controllers, and views required for Rails development.

[Goodness of scaffold that I personally felt]
(1) Not only the creation of the controller, but also 7 basic actions are automatically set.
In the controller created by scaffold, seven actions of "index", "show", "new", "edit", "create", "update", and "destroy" are automatically defined. As a result, you can quickly browse, create, modify, or delete data without having to set your own actions.

(2) The routing corresponding to the created controller is also set automatically.
routes after the scaffold command.In rb, "resources:The description "Controller name" has been added, and the routing corresponding to the newly created controller is automatically set.

③ All views corresponding to the actions defined in the controller are automatically generated.
When scaffold is used, all view files corresponding to the actions defined in the controller are also automatically generated.

④ The necessary model and migration file are also automatically generated.
In scaffold, if you specify "column name: data type" when entering a command, a model and migration file containing the specified information will be automatically generated.
Therefore, the rest is "$ rake db:You can easily create a database table just by entering the "migrate" command. (Details of this area will be described later)

→ Of course, I think it's a little different (or rather wasteful) to create an application to be released to the world using scaffold, but personally I make a prototype to implement the function I'm interested in I find it extremely convenient above.

So, the procedure for creating a Rails application using scaffold is described below.
It's really easy, so if you like, please try it out!

And finally, visualization using WordCloud

with open('textmining_source.txt') as f:
    text = f.read()

keywords = t.parse(text)

words = []
for x in keywords.split("\n"):
    word = x.split("\t")[0]
    if word == "EOS":
        break
    else:
        category = x.split("\t")[1].split(",")[0]
        if category == "noun":
                words.append(word)
        else:
            category = x.split("\t")[1].split(",")[0]
            if category == "adjective":
                words.append(word)

splitted =' '. join(words)
keywords = splitted.replace(',',' ')
 
wordcloud = WordCloud(
    background_color="white",
    stopwords={"thing","this","For","It","By the way","Yo","of","Etc.","thing"},
    font_path="NotoSansCJKjp-hinted/NotoSansCJKjp-Regular.otf",
    width=800,height=600).generate(keywords)
 
 
wordcloud.to_file("./textmining_result.png ")

WordCloud スクリーンショット 2019-11-19 0.01.02.png

By the way, the following is a visualization of Steve Jobs' famous speech in WordCloud. It seems that you can get a sense of the whole story just by looking at it!

Second story スクリーンショット 2019-11-19 0.36.18.png

Third story スクリーンショット 2019-11-19 0.34.46.png

Recommended Posts

[Note] WordCloud from morphological analysis
Collecting information from Twitter with Python (morphological analysis with MeCab)
[Python] Morphological analysis with MeCab
Japanese morphological analysis using Janome
Python: Japanese text: Morphological analysis
ChIP-seq analysis starting from zero
Natural language processing 1 Morphological analysis
Japanese morphological analysis with Python
[PowerShell] Morphological analysis with SudachiPy
From the introduction of JUMAN ++ to morphological analysis of Japanese with Python
100 language processing knocks 2020: Chapter 4 (morphological analysis)
100 Language Processing Knock 2020 Chapter 4: Morphological Analysis
Text mining with Python ① Morphological analysis
100 Language Processing Knock Chapter 4: Morphological Analysis
■ [Google Colaboratory] Use morphological analysis (janome)
■ [Google Colaboratory] Use morphological analysis (MeCab)
I played with Mecab (morphological analysis)!
[Language processing 100 knocks 2020] Chapter 4: Morphological analysis
100 Language Processing Knock 2015 Chapter 4 Morphological Analysis (30-39)
From preparation for morphological analysis with python using polyglot to part-of-speech tagging