Let's analyze Covid-19 (Corona) data using Python [For beginners]

There was a lecture video on overseas youtube that uses Python to analyze data using data such as the number of infected people of Covid-19 (Corona), so I tried it.

Click here for the source video ↓ Analyzing Coronavirus with Python (COVID-19) by NeuralNine on Youtube

Target

Especially recommended for Python beginners who want to get used to Pandas and want to do data analysis. The lecture is in English, but it is a very simple and polite explanation, so please take a look.

About this article

--I practiced the video while commenting in Japanese on the Jupyter notebook. It is recommended that you practice while actually watching the video, but for those who are not good at English or who want to understand the flow of data analysis, I wrote it so that you can get an image just by reading this article. --Although we use real data (* source will be described later), the data analysis process is not aimed at particularly sharp analysis results, but rather focuses on exercises (such as the Pandas library). .. --We are using the data up to May 1, 2020. (* Lecture video is data up to 3/22 at the time of shooting)

Exercise

Analyzing Coronavirus with Python (COVID-19) by NeuralNine on Youtube

Download the dataset from the following site HDX(HUMANITARIAN DATA EXCHANGE)

Use the following in the linked dataset

• time_series_covid19_confirmed_global.csv
• time_series_covid19_deaths_global.csv
• time_series_covid19_recovered_global.csv

It is the data of [Infection (confirmation) number, number of deaths, number of recovery].

The name of the data is long, so change it as follows.

• time_series_covid19_confirmed_global.csv → covid_confirmed.csv
• time_series_covid19_deaths_global.csv → covid_deaths.csv
• time_series_covid19_recovered_global.csv → covid_recovered.csv

Import library

import pandas as pd
import matplotlib.pyplot as plt

%matplotlib inline

Read the data

confirmed = pd.read_csv("covid19_confirmed.csv")
deaths = pd.read_csv("covid19_deaths.csv")
recovered = pd.read_csv("covid19_recovered.csv")

Display the infected person's data as a trial (* In the original video, it is the data of 3/22 at the time of shooting, but the following is until 5/1)

confirmed.head()

• Latitude (Lat), Longitude (Long)

This time, I don't need Province and Lat / Long so much, so I'll delete each column.

confirmed = confirmed.drop(['Province/State','Lat','Long'],axis=1)
deaths = deaths.drop(['Province/State','Lat','Long'],axis=1)
recovered = recovered.drop(['Province/State','Lat','Long'],axis=1)

Let's aggregate this data by Country / Region

confirmed = confirmed.groupby(confirmed["Country/Region"]).aggregate("sum")
deaths = deaths.groupby(deaths["Country/Region"]).aggregate("sum")
recovered = recovered.groupby(recovered["Country/Region"]).aggregate("sum")

confirmed.head()

Next, the date is the feature quantity, but this time we want to use the country as the feature quantity, so we will transpose the data (replace the matrix).

confirmed = confirmed.T
deaths = deaths.T
recovered = recovered.T

confirmed.head()

At this point, the data is ready. Let's move on to the calculation.

First, let's look at the changes in the number of infected people. The data required here is the difference in the number of infected people between the day and the day before.

new_cases = confirmed.copy()

for day in range(1,len(confirmed)):
    new_cases.iloc[day] = confirmed.iloc[day] - confirmed.iloc[day - 1]

View the data for the last 10 days

new_cases.tail(10)

Let's compare with infected person data

confirmed.tail(10)

For example, Afghanistan, Algeria, Argentina and the United Kingdom have a large number of infected people, but the number of newly infected people is still high.

In new_cases, we looked at the daily "increases" in the number of infected people, but next let's look at the "increase rate". (Increase in the day / Number of infected people in the previous day) * 100 can be used to increase the rate.

growth_rate = confirmed.copy()

for day in range(1,len(growth_rate)):
    growth_rate.iloc[day] = ( new_cases.iloc[day] / confirmed.iloc[day-1] ) * 100

growth_rate.tail(10)

By the way, the number of infected people (confirmed) is the so-called cumulative number, so let's get the current progressive number of infected people (Active) here. By subtracting [deaths and recovered] from [confirmed], it seems that [currently progressive number of infected people (Active)] can be calculated.

active_cases = confirmed.copy()

for day in range(0,len(confirmed)):
    active_cases.iloc[day] = confirmed.iloc[day] - deaths.iloc[day] - recovered.iloc[day]

Then, let's use the data of this currently progressive number of infected people active_cases to investigate the rate of increase in the number of people with ongoing infections again. By examining this, it seems that we can see if it is likely to converge.

overall_growth_rate = confirmed.copy()

for day in range(0,len(confirmed)):
    overall_growth_rate.iloc[day] = ((active_cases.iloc[day] - active_cases.iloc[day-1]) / active_cases.iloc[day-1]) * 100

overall_growth_rate.tail(10)

Increasing rate of ongoing infections on the last 10 days (2020/04 / 22-05 / 01) (China, Italy, USA, Japan)

• A little bit, the original is added to this part.

First of all, China, which is considered to be a corona-affected country, seems to have recently converged, so let's take a look at the data of the last 10 days of China.

overall_growth_rate['China'].tail(10)

4/22/20   -3.314528
4/23/20   -7.731583
4/24/20   -8.774704
4/25/20   -4.852686
4/26/20   -9.107468
4/27/20   -9.118236
4/28/20   -2.866593
4/29/20   -5.448354
4/30/20   -4.441777
5/1/20    -5.904523
Name: China, dtype: float64

It can be seen that the rate of increase is negative, that is, the number of people infected with the progressive tense is declining.

Next, what about Italy?

overall_growth_rate['Italy'].tail(10)

4/22/20   -0.009284
4/23/20   -0.790165
4/24/20   -0.300427
4/25/20   -0.638336
4/26/20    0.241859
4/27/20   -0.273319
4/28/20   -0.574599
4/29/20   -0.520888
4/30/20   -2.967790
5/1/20    -0.598714
Name: Italy, dtype: float64

The rate of decrease is less than 1%, so it is a slight decrease, but it has not increased.

overall_growth_rate['US'].tail(10)

4/22/20    3.470050
4/23/20    3.307839
4/24/20    2.102556
4/25/20    3.874078
4/26/20    2.536775
4/27/20    2.064644
4/28/20    2.166569
4/29/20    2.377575
4/30/20   -0.668941
5/1/20     2.583283
Name: US, dtype: float64

America is still increasing by a few percent.

overall_growth_rate['Japan'].tail(10)

4/22/20    2.512198
4/23/20    6.794937
4/24/20    3.868765
4/25/20    2.382691
4/26/20    0.401248
4/27/20    5.408526
4/28/20   -3.589182
4/29/20   -2.875120
4/30/20    0.755803
5/1/20    -2.884444
Name: Japan, dtype: float64

In Japan as well, it seems that it has been decreasing little by little recently, but it is on the increase.

Japan is a big deal, so let's take a look at the average rate of increase over the last 10 days.

overall_growth_rate['Japan'].tail(10).mean()

1.277542288600591

It's about 1%, but it seems to be increasing. Well, the number of infected people is still small in Japan, so it can be said that it is relatively suppressed.

From here, we will add visualization.

Let's look at the mortality rate first. Mortality is important because it is an indicator of the severity of the corona in each region.

First, the mortality data frame is similar to the previous procedure. Mortality is expressed in terms of death / infected.

death_rate = confirmed.copy()

for day in range(0,len(confirmed)):
    death_rate.iloc[day] = (deaths.iloc[day] / confirmed.iloc[day]) * 100

Next, calculate the number of beds you need (and, conversely, how likely you are to run out). We use the hospitalization rate "hospitalization", which is the percentage of infected people who need a hospital. I don't know the correct number, so I will use a temporary number (0.05 in this case). You can change it to any number you like. In this lecture, we will focus on analysis and calculation methods, so we will leave the accuracy aside.

• By the way, the hospitalization rate is for those who are positive for corona and need to be hospitalized, and even if the remaining 95% are positive, we consider that hospitalization (bed) is not necessary.

hospitalization_rate_estimate = 0.05

hospitalization_needed = confirmed.copy()

for day in range(0,len(confirmed)):
    hospitalization_needed.iloc[day] = active_cases.iloc[day] * hospitalization_rate_estimate

hospitalization_needed.tail()

Even if you take out one country here, it is difficult to understand how serious it is, so let's look at the average of the last 5 days.

hospitalization_needed.tail().mean().mean()

532.5691978609626

Average number of beds required in all countries over the last 5 days. Of course there are variations, so it is not a very ideal reference value, but I will refer to this once. Let's take a look at the average of the last 5 days in Italy.

hospitalization_needed['Italy'].tail().mean()

5181.6900000000005

In other words, the figure was roughly 10 times the average in the world. It's pretty serious.

Visualize. However, there are too many countries, so let's choose some arbitrary countries this time. Here, select Italy, USA, China, Japan, Russia Spain.

countries = ['Italy','US',"China","Japan","Russia","Spain"]

ax = plt.subplot()
ax.set_facecolor("black")
ax.figure.set_facecolor("#121212")
ax.tick_params(axis="x",colors="white")
ax.tick_params(axis="y",colors="white")
ax.set_title("covid-19 confirmed by countries",color="white")

for country in countries:
    confirmed[country].plot(label=country)
plt.legend(loc="upper left")
plt.show()

The number of infected people in the US has increased significantly since the end of March. Let's see the number of deaths.

The shape of the graph does not change much, and it seems to be associated with the number of infected people.

Next, let's plot the rate of increase in infected people.

But now we're plotting on a bar chart. Also, in a bar chart, if the graph overlaps too much on one figure, the visibility will be low, so it will be displayed separately.

for country in countries:
    ax = plt.subplot()
    ax.set_facecolor("black")
    ax.figure.set_facecolor("#121212")
    ax.tick_params(axis="x",colors="white")
    ax.tick_params(axis="y",colors="white")
    ax.set_title(f"covid-19 confirmed growth rate {country}",color="white")
    growth_rate[country].plot.bar() 
    plt.show()

In the same way, let's look at the number of deaths and the mortality rate (* The above is the "infection rate increase rate" and this is the "mortality rate").

ax = plt.subplot()
ax.set_facecolor("black")
ax.figure.set_facecolor("#121212")
ax.tick_params(axis="x",colors="white")
ax.tick_params(axis="y",colors="white")
ax.set_title("covid-19 deaths by countries",color="white")

for country in countries:
    deaths[country].plot(label=country)
plt.legend(loc="upper left")
plt.show()

for country in countries:
    ax = plt.subplot()
    ax.set_facecolor("black")
    ax.figure.set_facecolor("#121212")
    ax.tick_params(axis="x",colors="white")
    ax.tick_params(axis="y",colors="white")
    ax.set_title(f"covid-19 deaths rate {country}",color="white")
    death_rate[country].plot.bar() 
    plt.show()

You can see that the mortality rate varies by country.

Finally, let's move on to simulating the effects of the coronavirus in the future. As a tentative value, let's assume that the number of infected people increases by 1% on a daily basis.

simulated_growth_rate = 0.01

Now add the upcoming new date data for your forecast. Specify the range and use the date_range method that can generate date data. The last data used this time is 05/01/20, so it will be 40 days from the next day.

dates = pd.date_range(start="05/02/2020",periods=40,freq='D')
dates = pd.Series(dates)
dates = dates.dt.strftime("%m/%d/%Y")

simulated = confirmed.copy()
simulated = simulated.append(pd.DataFrame(index=dates))

for day in range(len(confirmed),len(confirmed)+40):
    simulated.iloc[day] = simulated.iloc[day-1] * (1 + simulated_growth_rate)
ax = plt.subplot()
ax.set_facecolor("black")
ax.figure.set_facecolor("#121212")
ax.tick_params(axis="x",colors="white")
ax.tick_params(axis="y",colors="white")
ax.set_title(f"covid-19 future for Japan",color="white")
simulated['Japan'].plot()
plt.show()

• As a reminder, this is a number based on a tentative growth rate (* it continues to increase by 1% daily). As a rigorous simulation, you don't have to take it. that's all. There are some differences from the original video, but I think I got a rough idea of the flow of data analysis. Please take a look at the original video. At the end of the video, the poster, Neural Nine, emphasized that this analysis has tentative numbers, so you don't have to take it seriously. He told me that what I should do is important.