[PYTHON] Plotly Dash on Google Colab

Dash (https://dash.plotly.com/) is a framework for data visualization in python.

I thought this would be available on Google Colab, but I found a good article that has already been explained. https://qiita.com/OgawaHideyuki/items/725f4ffd93ffb0d30b6c

So, this article is a record that I used it as a reference and moved my hand. The theme is to easily visualize the number of people infected with corona in each country. Let's display a map and a time series graph.

** 2020-12-27 Addendum: ** Added map display.

Environment and resources to use

Use Google Colaboratory.

Various resources

• Dash https://dash.plotly.com/
• A Python framework that makes it easy to create interactive data visualization screens using the graph libraries Plotly and Flask.
• ** Infected person data by country/region ** https://github.com/CSSEGISandData/COVID-19
• Uses Johns Hopkins University Infected Persons Data by Country / Region.
• Mapbox　https://www.mapbox.com/
• A map service used to display scatter plots on a map.

This notebook

Since the map scatter plot display uses Mapbox tokens and Google drive, the notebook is divided into two parts, chronological and map.

• ** Time series graph display **
  • https://colab.research.google.com/drive/1fUP4818fSsFFFlUHlLGNoTxq8uoL2VAu?usp=sharing
• ** Map scatter plot display **
  • https://colab.research.google.com/drive/1MTECSFu3lrl7w0_NHZXYp8ENPYHu4dEg?usp=sharing

Time series graph display

Preparation and data

First, install the package for using Dash from your Google Colab/Jupyter notebook.

! pip install jupyter_dash
! pip install --upgrade plotly

Import the packages associated with Dash.

import dash 
from jupyter_dash import JupyterDash 
import dash_core_components as dcc 
import dash_html_components as html 
import plotly.express as px
from dash.dependencies import Input, Output

Get corona infection data from GitHub. For the data, please refer to the following page. https://dodotechno.com/covd-19-visualization/

! wget https://github.com/CSSEGISandData/COVID-19/raw/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_confirmed_global.csv

Load the downloaded csv into the data frame.

import pandas as pd
df = pd.read_csv("time_series_covid19_confirmed_global.csv")

Aggregate by region, delete latitude and longitude, transpose to column by country, and add date column.

df = df.groupby(['Country/Region'], as_index=False).sum()
df.drop(["Lat","Long"], axis=1,inplace=True)
df = df.T
df.columns =  df.iloc[0]    
df = df[1:]
df.reset_index(inplace=True)
df.rename(columns={'index': 'date'},inplace=True)
df

The resulting table looks like this:

Try to display it in a notebook

First, let's graph the number of infected people in Japan. The horizontal axis is the date and the vertical axis is the number of infected people, and you can see an increase that seems to be the first wave (late April), the second wave (early August), and the third wave (November).

px.line(df, x="date", y="Japan")

Next, let's make any country selectable in the dropdown. You can select it on your notebook, so give it a try. If the runtime is stopped, try selecting "Runtime"-> "Run All Cells" from the menu. https://colab.research.google.com/drive/1fUP4818fSsFFFlUHlLGNoTxq8uoL2VAu#scrollTo=Kr-FsvLIpCoN&line=1&uniqifier=1

app = JupyterDash(__name__)

app.layout = html.Div([
  dcc.Dropdown(id="my_dropdown",
    options=[{"value": country, "label": country} for country in df.columns.unique()],
    value=["Japan"],
    multi=True
    ),
  dcc.Graph(id="my_graph")
])

@app.callback(Output("my_graph", "figure"), Input("my_dropdown", "value"))
def update_graph(selected_country):
  return px.line(df, x="date", y=selected_country)

app.run_server(mode="inline")

From the dropdown, select Japan and Canada to display. It seems that Canada is also on the rise.

Next, when I add the United States, it's not really comparable to Japan. .. After all there is an impact when looking at the graph. I want the vaccine to be effective (though not for each person).

Display on map

Let's display a scatter plot on the map using the same data.

Preparation and data

We use a map service called Mapbox. An access token is required to use it. If you do not have an account, sign up below to get an access token. https://account.mapbox.com/ At a minimum, you only need your ID, password, and email address.

This time we will store the token in Google drive. If you want to run it yourself, you can also embed it in your code as a string.

Here, as an example, upload the text file mapbox-token.txt with the contents of the Mapbox token pasted directly under My Drive on Google drive.

Mount Google drive and load the Mapbox token. The OAuth token at the time of mounting is displayed when you jump to the page of the URL displayed at the time of execution, so enter it by copy and paste.

from google.colab import drive
drive.mount('/content/drive')

f = open('/content/drive/My Drive/mapbox-token.txt', 'r')
MAPBOX_TOKEN = f.read()
f.close()

Importing Jupyter dash is similar to a time series graph.

! pip install jupyter_dash
! pip install --upgrade plotly

import dash 
from jupyter_dash import JupyterDash 
import dash_core_components as dcc 
import dash_html_components as html 
import plotly.express as px
from dash.dependencies import Input, Output

Corona infected person data is also acquired.

! wget https://github.com/CSSEGISandData/COVID-19/raw/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_confirmed_global.csv

Load the downloaded csv into the data frame.

df_map = pd.read_csv("./time_series_covid19_confirmed_global.csv")
df_map

This time, we will only collect data for each region, leaving the latitude and longitude and not transposing.

df_map = df_map.groupby(['Country/Region'], as_index=False).sum()
df_map

Now that we are ready, we will display it on a map.

Display on map

The date and color specification criteria are selectable and displayed. I refer to this. https://qiita.com/banquet_kuma/items/e02ba60661cf91af37de

• color_opt and date_opt exclude latitude / longitude and latitude / longitude + country name from the column names in the data frame.
• MAPBOX_TOKEN is the value read from the file uploaded to Google drive earlier. If you want to execute it at hand, you can paste it as it is as a character string.
• The notebook code should be able to access the drive of the google account that runs it, but I haven't tried it outside of my account, so please point it out if it doesn't work.
• You can change the appearance in various ways. The options will be described later.

app = JupyterDash(__name__)

color_opt = [dict(label=x, value=x) for x in df_map.columns]
del color_opt[2]
del color_opt[1]
date_opt = color_opt.copy()
del date_opt[0]

app.layout = html.Div(
    [
        html.Div(
            [
                html.P(["date:", dcc.Dropdown(id='date', options=date_opt)]),
                html.P(["color:", dcc.Dropdown(id='color', options=color_opt)]),
            ],
            style={"width": "20%", "float": "left"}
        ),
        dcc.Graph(id="graph", style={"width": "80%", "display": "inline-block"}),
    ]
)

@app.callback(Output("graph", "figure"), [Input("date", "value"), Input("color", "value")])
def update_graph(date, color):
  px.set_mapbox_access_token(MAPBOX_TOKEN)
  if not color:
    color = date
  return  px.scatter_mapbox(df_map,
                        lat="Lat",
                        lon="Long",
                        color=color,
                        size=date,
                        size_max=20,
                        zoom=0,
                        center={'lat': 35, 'lon': 135},
                        title="Number of people infected with corona in each country",
                        color_continuous_scale=px.colors.diverging.BrBG,
                        hover_name=date)

app.run_server(mode="inline")

When executed, it looks like this. The display is beautiful.

Select a date to see the cumulative number of infected people at that time. 2020-12-01 looks like this.

The display is done by a function called plotly.express.scatter_mapbox. I couldn't find a Japanese explanation for the parameters of plotly.express.scatter_mapbox, so I'll give a brief explanation for your reference (if you know a good source, please let me know in the comments).

https://plotly.github.io/plotly.py-docs/generated/plotly.express.scatter_mapbox.html

• ** data_frame ** (DataFrame or array-like or dict) – A DataFrame for passing data by column name.
• ** lat ** (str or int or Series or array-like) – Latitude on the map.
• ** lon ** (str or int or Series or array-like) – Longitude on the map.
• ** color ** (str or int or Series or array-like) – A specification for assigning colors to markers on the map.
• ** text ** (str or int or Series or array-like) – A text label for display in figures.
• ** hover_name ** (str or int or Series or array-like) – Tooltip display name when hovering the mouse.
• ** hover_data ** (list of str or int, or Series or array-like, or dict) – Tooltip data when hovering the mouse. You can specify two elements, the first element specifying the display True (default)/False or the data display format. The second element is the value to display.
• ** custom_data ** (list of str or int, or Series or array-like) – Additional data for use in widgets and Dash callbacks. It's not visible to the user, but it's used for things like figure events (lasso tool, note: maybe range selection).
• ** size ** (str or int or Series or array-like) – The size of the mark on the map.
• ** animation_frame ** (str or int or Series or array-like) – Used to assign marks to animation frames (Note: I'm not sure about animation frames ...).
• ** animation_group ** (str or int or Series or array-like) – Provides an invariant object throughout the animation frame, treating the same animation_group to describe the same object in each frame.
• ** category_orders ** Specifies the order of the axis, legend, and facet category values ​​(default is {}). Order is not guaranteed in python 3.6 and earlier.
• ** labels ** (dict with str keys and str values ​​(default {})) – By default column names are used in axis titles, legend entries, and hover diagrams. The dict that corresponds to the column name that overrides this.
• ** color_discrete_sequence ** (list of str) – Color cycle specification. Specify a valid color as CSS.
• plotly.express.colors Especially plotly.express.colors.qualitative` has a useful color sequence.
• ** color_discrete_map ** (dict with str keys and str values ​​(default {})) – A valid CSS color specification column.
• ** color_continuous_scale ** (list of str) – Specifies the color scale when the column specified by color is continuous numeric data.
• Convenient color scales plotly.express.colors Especially available in plotly.express.colors.sequential, plotly.express.colors.diverging and plotly.express.colors.cyclical.
• ** range_color ** (list of two numbers) – Overrides consecutive color scale specifications when specified.
• ** color_continuous_midpoint ** (number (default None)) – Specifies the midpoint of the continuous color scale. Recommended when using plotly.express.colors.diverging.
• ** opacity ** (float) – Specifies the transparency of the marker.
• ** size_max ** (int (default 20)) – Specifies the maximum size of the marker.
• ** zoom ** (int (default 8)) – Specify the zoom size of the map from 0 to 20.
• ** center ** (dict) – Specifies the center of the map in latitude and longitude (lat, lon-keyed dict).
• ** mapbox_style ** (str (default'basic', needs Mapbox API token)) – Specifies the style of the map. Some styles require a Mapbox API token.
• 'open-street-map','white-bg','carto-positron',' carto-darkmatter','stamen-terrain','stamen-toner','stamen-watercolor' do not require tokens is.
• 'basic','streets','outdoors','light','dark','satellite','satellite- streets' require tokens.
• ** title ** (str) – Specify the title.
• ** template ** (str or dict or plotly.graph_objects.layout.Template instance) – Specify the template name or definition.
• ** width ** (int (default None)) – Specify the width in pixels.
• ** height ** (int (default None)) – Specify the height in pixels.

There are many. ..

The remaining challenges

Here are some issues.

• I'm glad to select the sliver bar for the date.
• Maybe because the number of digits of the value is different, there are countries that can hardly be seen, so adjustment is necessary (such as logarithmic output?).
• Tooltips are not displayed. Cause unknown.

in conclusion

It's easy to implement with a little visualization, and it's easy to publish on the Internet with Google Colab. Processing csv may be the most troublesome.

I would like to add visualization on the map. (→ 2020-12-27 Map display added, there are remaining issues) It seems that we can still visualize various things, so we may do it soon.

I found out by running it, but the graph does not remain when Google Colab is stopped, so if you need to publish it, you may want to use other methods (matplotlib, plotly, etc.) for Heroku or notebooks. Maybe.

Related / reference URL

Dash related

• Dash　https://dash.plotly.com/
• Jupyter_dash that can use Dash on Jupyter https://qiita.com/OgawaHideyuki/items/725f4ffd93ffb0d30b6c

Data related

• COVID-19 Data Repository by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University　　https://github.com/CSSEGISandData/COVID-19
• I tried to visualize the number of corona infected people with Python + Matplotlib. (As of 4/4) https://dodotechno.com/covd-19-visualization/

Mapbox display related

• Mapbox　https://www.mapbox.com/
• Visualize vehicle data acquired from OBD with Dash and Plotly Express Part 2 https://qiita.com/banquet_kuma/items/e02ba60661cf91af37de
• plotly.express.scatter_mapbox　https://plotly.github.io/plotly.py-docs/generated/plotly.express.scatter_mapbox.html