[PYTHON] Dynamically display epidemic data using the Grafana Dashboard
Learn how Alibaba Cloud dynamically displayed information about COVID-19 outbreaks using the Grafana Dashboard.
- This blog is a translation from the English version. You can check the original from here. We use some machine translation. We would appreciate it if you could point out any translation errors. *
Project preparation
I want to customize the dashboard that displays the epidemic status to clarify the purpose. Since I am currently in Shanghai, I would like to display the number of confirmed cases, suspected cases, recoverers, and deaths nationwide, and at the same time, display the case curve to observe the trend of epidemic development, and each province and district in Shanghai. I want to display the status of.
Grafana is just a tool for viewing data, so you need to get the data source first. Currently, the available plague data sources cannot be integrated directly into Grafana. Therefore, you need to do the following:
--Prepare the ** Grafana program ** on your laptop or Kubernetes cluster. We recommend using a Docker container to run Grafana.
--Install the ** SimpleJson plugin ** that can convert JSON format data to a data source for Grafana.
Data source development
Here, I am developing a data source using Python Bottle. You can also choose Flask (https://flask.palletsprojects.com/en/1.1.x/?spm=a2c65.11461447.0.0.2e7418faOcjQf4), which has the same functionality. I decided to use Bottle because I used it to develop my previous Grafana data source. Not only debug settings, but also Dockerfile for building Docker images and deploy.yaml file for deploying Kubernetes are prepared, and they are ready to use. Developing a Grafana data source using Python is very easy. All you have to do is make sure your data meets the format requirements of SimpleJson. Oz Nahum Tiram's blog on how to use Python to develop data sources for Grafana. Article "[Visualize Almost Anything with Grafana and Python](http://oz123.github.io/writings/2019-06-16-Visualize-almost-anything-with-Grafana-and-Python/index.html?spm=" Please read a2c65.11461447.0.0.2e7418faOcjQf4) ".
There are two types of data available for data source customization:
Time series type data
To view epidemic trends in China, especially Shanghai, in real time, you can view confirmed cases, suspected cases, collections, deaths, and changes in data compared to previous data. You can also draw curves to compare confirmed and suspected cases, and the number of recoveries and deaths.
For confirmed cases, simply return the data by combining the number of confirmed cases nationwide gntotal with the current time stamp. Other metrics can be processed in the same way.
@app.post('/query')
def query():
print(request.json)
body = []
all_data = getDataSync()
time_stamp = int(round(time.time() * 1000))
for target in request.json['targets']:
name = target['target']
if name == 'gntotal':
body.append({'target': 'gntotal', 'datapoints': [[all_data['gntotal'], time_stamp]]})
body = dumps(body)
return HTTPResponse(body=body, headers={'Content-Type': 'application/json'})
Table type data
You can use a table that maps the number of confirmed cases, suspected cases, collections, and deaths in each province of China, and another table that shows the same data for each district in Shanghai.
You can extract names, confirmed cases, suspected cases, recoverers, and dead from the data and add them to the row rows.
@app.post('/query')
def query():
print(request.json)
body = []
all_data = getDataSync()
sh_data = getShDataSync()
if request.json['targets'][0]['type'] == 'table':
rows = []
for data in all_data['list']:
row = [data['name'], data['value'], data['susNum'], data['cureNum'], data['deathNum']]
rows.append(row)
sh_rows = []
for data in sh_data['city']:
row = [data['name'], data['conNum'], data['susNum'], data['cureNum'], data['deathNum']]
sh_rows.append(row)
bodies = {'all': [{
"columns": [
{"text": "Province", "type": "name"},
{"text": "确 诊", " type": "conNum"},
{"text": "pseudo", " type": "susNum"},
{"text": "Cure", "type": "cureNum"},
{"text": "death", "type": "deathNum"}
],
"rows": rows,
"type": "table"
}],
'sh': [{
"columns": [
{"text": "Province", "type": "name"},
{"text": "确 诊", " type": "value"},
{"text": "pseudo", " type": "susNum"},
{"text": "Cure", "type": "cureNum"},
{"text": "death", "type": "deathNum"}
],
"rows": sh_rows,
"type": "table"
}]}
series = request.json['targets'][0]['target']
body = dumps(bodies[series])
return HTTPResponse(body=body, headers={'Content-Type': 'application/json'})
Panel type selection
Data is generally displayed using four panels. --Single stat is used to display the case number.
--The graph displays a comparison curve.
--Table is used to display the table.
--The text is used for the title of the text.
Data source settings
Now let's set up the data source.
Case number panel
Since there is only one value here, select First.
Case number graph
These graphs compare the number of confirmed and suspected cases, the number of collections and the number of deaths.
Data table
result
The overall finish is fine and is used as the company's trendy dashboard. Here, we are using a relatively small screen Xiaomi TV, so we have expanded the fonts and display panel to enhance the demo.
building
After packaging your code into a Docker image, you can run your code in any environment or in a Kubernetes cluster. The image has been uploaded to Docker Hub and can be pulled from there for immediate use.
# Dockerfile
FROM python:3.7.3-alpine3.9
LABEL maintainer="[email protected]"
COPY . /app
RUN echo "https://mirrors.aliyun.com/alpine/v3.9/main/" > /etc/apk/repositories \
&& apk update \
&& apk add --no-cache gcc g++ python3-dev python-dev linux-headers libffi-dev openssl-dev make \
&& pip3 install -r /app/requestments.txt -i http://mirrors.aliyun.com/pypi/simple --trusted-host mirrors.aliyun.com
WORKDIR /app
ENTRYPOINT ["uwsgi","--ini","uwsgi.ini"]
Running it
-** Pull the image. ** **
docker pull guoxudongdocker/feiyan-datasource
-** Execute the image. ** **
docker run -d --name datasource -p 8088:3000 guoxudongdocker/feiyan-datasource
-** Add a data source. ** **
Select the SimpleJson data source, click Add and enter the address of the data source.
-** Import dashboard **
Click on the ʻUpload.json file and select wuhan2020-grafana / dashboard.json`.
-(Optional) Use Kubernetes for deployment.
kubectl apply -f deploy.yaml
Overview
As of mid-February, when I wrote this, the number of confirmed cases in China is still increasing rapidly, but fortunately the growth rate of suspected cases is beginning to decline. It can be seen that the rate of increase in the number of confirmed cases is steadily increasing, and the number of collections is also increasing. Compared to other regions, the number of cases in Shanghai has not increased significantly, even though many people have returned to work. As part of this, strict crackdowns on residential areas are beginning to take effect. Currently, only one person has died in Shanghai, as it recorded China's first recovery. In general, as long as we pay attention to prevention and stay at home, there is no doubt that we will overcome the epidemic and overcome this difficult time.
The JSON file for importing dashboards and the YMAL file for deploying Kubernetes are on GitHub. The address of the project is here. https://github.com/sunny0826/wuhan2020-grafana
While continuing to fight the coronaviruses that are occurring around the world, Alibaba Cloud wants to do its part and do everything it can to help others in the fight against the coronavirus. I am. For information on how to support your business continuity, see https://www.alibabacloud.com/campaign/fight-coronavirus-covid-19 Please check with -covid-19).
- Alibaba Cloud is the No. 1 (2019 Gartner) cloud infrastructure operator in the Asia Pacific region with two data centers in Japan and more than 60 availability zones in the world. Click here for more information on Alibaba Cloud. Alibaba Cloud Japan Official Page *
Recommended Posts