[PYTHON] That's why I'll look up tweets from the House of Representatives election

It's been a year since the ban on online elections was lifted, and every time the elections are over, it's a speech space that keeps reducing the remaining machines of democracy and Japan like a spelunker. How are you doing today?

By the way, this time I will examine the contents of the tweets before and after the ballot counting.

Collection contents

2014/12/14 From 18:00 to 07:00 Collect tweets containing the following information

#election,#House of Representatives election,election

** Keep getting tweets containing specific keywords using Streaming API in Python ** http://qiita.com/mima_ita/items/ecdf7de2fe619378beee

Code used for collection and analysis

https://github.com/mima3/stream_twitter

Confirmed to work on Windows7 Python 2.7

Analysis of collection results

The collected data can be downloaded from the following. http://needtec.sakura.ne.jp/doc/shuin47twitter.zip

Histogram by time

First, let's look at the number of tweets by time. Let's look at the hourly data from 18:00 to 07:00 on 2014/12/14 using the above code.

python twitter_db_hist.py "2014/12/14 9:00" "2014/12/14 22:00" 3600

• Because the time on Twitter can be obtained in UTC, it is 9 hours off in Japan time.

The result is as follows:

The timing of ballot counting at 20:00 is the highest. And the number of tweets will decrease over time. However, it revived in the 1:00s, and after that, the number of tweets decreased, and it recovered a little from around 5:00 when I woke up.

It's understandable that the number of tweets decreases at midnight and increases in the morning. But why did the number of tweets increase in the midnight 1:00s?

For this reason, let's take a look at the 1:00s in 1-minute units.

python twitter_db_hist.py "2014/12/14 16:00" "2014/12/14 17:00" 60

Looking at this result, it shows a rapid excitement around 1:27 minutes.

What happened at this time? Here, let's check the threads of the people of the Kaieda Research Institute who love the Democratic Party.

** [Fukatsu no Jumon is different] Kaieda Democratic Party Research 802th [Tosen no Sho has been decided] ** http://anago.2ch.net/test/read.cgi/asia/1418565521/

811: Nameless place of sunrise: 2014/12/15(Month) 01:26:44.86 ID:tG+ZZ8gB
[Banri Kaieda] Democratic Party representative Banri Kaieda did not revive in proportional Tokyo block, and the election was confirmed (01):19）(c)2ch.net
    http://daily.2ch.net/test/read.cgi/newsplus/1418574054/

812: Nameless place of sunrise: 2014/12/15(Month) 01:26:49.76 ID:4Us97nfn
Lost decision w

813: Nameless place of sunrise: 2014/12/15(Month) 01:26:51.11 ID:pW7uplw3
Goodbye, Mari

814: Nameless place of sunrise: 2014/12/15(Month) 01:27:01.20 ID:yIjazH47
Wow ah ah NHK also lost w

815: Nameless place of sunrise: 2014/12/15(Month) 01:27:02.02 ID:NOhUWn58
Mali completely defeated at NHK

No, I'm going to Phoenix from here! It's definitely Fenix, so _____

816: Nameless place of sunrise: 2014/12/15(Month) 01:27:08.70 ID:4zmUGrZE
    >>802
㌧. I haven't bought any snacks so I can open the mackerel can w

Looking at the situation at that time, it seems that Asahi reported the extraordinary defeat of Kaieda at 1:19, and NHK also reported the news at 1:27.

As expected, it can be said that the advancement and retreat of the leader of the first opposition party had the impact of wiping out the drowsiness of Twitter at midnight. The results also show that television has stronger diffusion power than newspaper extras.

Extraction of frequent words

Next, let's look at frequent words. By using Mecab, we performed morphological analysis and aggregated the words.

This can be done with the following script.

python twitter_db_mecab.py "2014/12/14 9:00" "2014/12/14 22:00" > mecab.txt

The best 100 are displayed below.

After all, the most extracted party name was the "Liberal Democratic Party," which took the majority. Next is the "Democratic Party," followed by the "Next Generation," followed by the "Renewal" and the "Communist Party." The next-generation party seems to have a considerable gap between the actual number of seats and the degree of attention on the Internet.

"Tokyo" and "Okinawa" were extracted by place name. As for Tokyo, there was a retweet of the article of "Tokyo Shimbun", so it was extracted a lot, and for Okinawa, the LDP was in the form of annihilation of the single-seat constituencies, so it is thought that it attracted more attention than other areas.

The names of people that attracted attention were "Abe," "Beat Takeshi," and "Kaieda." Not to mention the prime minister and the leader of the first opposition party, it was surprising that "Beat Takeshi" was noticed. Apparently this is because "Beat Takeshi" was appearing on Nico Nico Live.

Exploring the relationship between clauses

Finally, let's use Cabocha to tabulate the relationship between clauses. See below for how to install Cabocha on Windows.

** Put Cabocha in Windows and analyze the dependency with Python ** http://qiita.com/mima_ita/items/161cd869648edb30627b

This time, I analyzed with 0.66. I think the latest results will be similar.

This can be done with the following script.

python twitter_db_cabocha.py "2014/12/14 9:00" "2014/12/14 22:00"  > cabocha.txt

The best 100 are displayed below.

Lost → As certainty is ranked first, Twitter seems to be more interested in who will be dropped than who will be accepted.

Also, it seems that there are many references to youth turnout. However, it is also true that there are many news stories that "the turnout of teens is 0%".

"High"-> "Japan" seems to be the result of a large number of tweets saying "Only Japan with a high level of education uses the method of writing names on ballots."

Also, as I mentioned at the beginning, I investigated democracy and Japan, where the remaining opportunities are reduced at each election. Of the two tweets saying that democracy will die, there are 11 tweets saying that democracy will not die, so it seems that democracy's remaining opportunities are not less than expected.

However, a few words were extracted to reduce the remaining machines, such as the following.

How the reduction of the remaining machines of Japan is as follows.

From this result, it seems that the number of remaining democratic aircraft has decreased to 9 in this election, and the number of remaining aircraft of Japan has decreased by about 23.

Summary of what can be read from the data

・ When Mr. Kaieda becomes unemployed, he gets excited even though it is midnight, and when he sees that the phrase of defeat → certainty appears frequently, it is more noticeable who will fall than who will accept it.

・ Looking at the number of appearances of the word “next generation” and the actual results, it can be said that seats cannot be taken because of the attention on the Internet.

・ Democracy and Japan had the image that the number of remaining aircraft is decreasing every election, but it seems that this was not the case.

Real summary

You can do a bogus analysis like that. For tweets by hour, as in this example, I think you should look at the increase and decrease of the data and investigate in detail where there was a change.

It's easy to see that the word frequency is certainly getting a lot of attention. However, it should be noted that a large number of extracts for this example is not always a positive reaction.

With regard to parsing, it may be possible to overcome the weaknesses of examining frequent word-only occurrences. But honestly, I wouldn't have fully realized that possibility this time around. This will be an issue for the future.

By the way, I couldn't get the tweet of the key mark with the filter of Streaming API.