[PYTHON] I tried to display the point cloud data DB of Shizuoka prefecture with Vue + Leaflet
What I made this time
Shizuoka Point Cloud Data Map
GitHub - shizokaPCDB
Caution
This article is not an article that displays lidar data in Leaflet. It displays information in the point cloud database (data acquisition location, construction date, etc.).
Introduction
Rock Shizuoka Prefectural Office has a large number of LIDAR data of construction products as open data on its own website Shizuoka Point Cloud DB (hereinafter referred to as "Shizuoka PCDB"). It is open to the public. Looking at the source with interest, I found that it seems that data can be obtained from the outside (because JavaScript was written directly in html). However, as you can see from this site and the above image, (1) the number of markers is large and the performance is affected, and (2) every time the map area changes, the data contained in that area is acquired from the server, and the performance further deteriorates. I thought there were two problems: what I was doing. I thought that there was room for improvement if the data could be obtained from the outside, so I decided to develop it.
For that reason, we will send it in two ways: access to the database, shaping, and display at the front desk.
Technology used
- Flask
- Vue.js
- Vue2-leaflet
Database access / formatting
access
Looking at the source of Shizuoka PCDB, access to the server and display in Leaflet were implemented in about 100 lines. The part that retrieves data from the server is shown below.
var data = { request : "MarkerSet",
Xmax : map.getBounds().getNorthEast().lng,
Xmin : map.getBounds().getSouthWest().lng ,
Ymax : map.getBounds().getNorthEast().lat,
Ymin : map.getBounds().getSouthWest().lat };
$.ajax("ankenmapsrc",{
type: "GET",
data: data,
success: function(data, dataType){
var myIcon = L.icon({
iconUrl: 'http://cdn.leafletjs.com/leaflet-0.5/images/[email protected]',
iconSize: [15,25],
iconAnchor: [5, 5],
});
//The following processing continues
You can get the data by accessing the ankenmapsrc with the query data. Here, data has the request name "MarkerSet" and Xmax, Xmin, Ymax, Ymin indicating the area to be acquired. The four variables in the area store the area displayed in Leaflet. This process is called every time the Leaflet display area is updated. In other words, the data is acquired from the server each time.
Now let's look at the return value of this request.
30XXX01010001:2018 Nirayama Reverberatory Furnace Measurement Business:138.96214537214:35.03962001009?
28XXX00030007:Shiraito no Taki Takimi Bridge area maintenance project No. 7:138.58870495572:35.312506370532?
28XXX00030008:Shiraito no Taki Takimi Bridge area maintenance project No. 8:138.58881502806:35.312596432406?
28XXX00030009:Shiraito no Taki Takimi Bridge area maintenance project No. 9:138.58892510063:35.312686494178?
29C2001011361:2017 [29th-Ｃ２００１-No. 01] Outsourced fact-finding survey of outdoor advertisements on the Izu Peninsula (Kannami Town Road)_1-Line 2):138.93794860595:35.083520492945
...
Originally there is no line break, but it is added for explanation. Looking at this data, you can see that the delimiter for each construction is "?" And the delimiter for each data element is ":".
Plastic surgery
Access Shizuoka PCDB with Flask and create an API server that responds to the formatted data.
import urllib.request, urllib.parse
import json
@app.route('/markers')
def getMarkers():
#In order to get all records, set the latitude and longitude including the entire Shizuoka prefecture as an integer value.
xMax = 140
xMin = 137
yMax = 36
yMin = 33
params = {
'request':'MarkerSet',
'Xmax':xMax,
'Xmin':xMin,
'Ymax':yMax,
'Ymin':yMin
}
p = urllib.parse.urlencode(params)
url = "https://pointcloud.pref.shizuoka.jp/lasmap/ankenmapsrc?" + p
#Request to SIZUOKA POINT CLOUD DB with the URL parameter generated above and get the matter list character string
allAnkenStr = ""
with urllib.request.urlopen(url) as res:
allAnkenStr = res.read().decode()
#Create json to return
ankensObj = {
"ankenList":[]
}
ankenList = allAnkenStr.split('?')
for anken in ankenList:
ankenInfo = anken.split(':')
#If there is inappropriate data, skip it
if len(ankenInfo) != 4:
continue
#Convert Japanese calendar to Western calendar
yy = int(ankenInfo[0][:2])
#Reiwa
if yy < 24:
yyyy = 2018 + yy
else:
yyyy = 1988 + yy
ankenObj = {
"no":ankenInfo[0],
"name":ankenInfo[1],
"lon":ankenInfo[2],
"lat":ankenInfo[3],
"year":yyyy
}
ankensObj['ankenList'].append(ankenObj)
return jsonify(ankensObj)
Problems explained at the beginning (2) Every time the map area changes, the data contained in that area is acquired from the server, and the performance is further reduced. It was solved by doing. That's because the total number of cases was about 1400, and it wasn't too late to actually get all of them, so we concluded that we should get all of them first.
We decided to parse the acquired anken data and return the construction number, construction name, longitude / latitude, and construction year as json.
Display at the front
Remaining problems (1) The number of markers is large and the performance is affected, but I thought that it would be solved if it was displayed on Mapbox GL JS, which operates quickly. However, when I implemented it, the marker display became slower than Leaflet. So, I solved this problem by adopting Leaflet's Marker Cluster.
Regarding the environment construction of Leaflet and Vue, Try # 027 – I tried to build the development environment of Leaflet and Mapbox GL JS with Vue.js I will omit it because it is detailed. In addition, install vue2-leaflet-markercluster.
npm install vue2-leaflet-markercluster
Asynchronous communication with API server with Vue
Use the Fetch API. Get the data with App.vue and pass it to MapPane.vue.
App.vue
<MapPane :ankens="ankens"/>
App.vue
data() {
return {
ankens:[],
}
},
created() {
let vm = this
fetch("/markers")
.then(response => {
return response.json()
})
.then(data => {
//After sorting, I'm passing data
vm.ankens = data.ankenList.sort(function (a, b) {
if (a.no < b.no) {
return 1
}
if (a.no > b.no) {
return -1
}
return 0
}).sort(function (a, b) {
if (a.year < b.year) {
return 1
}
if (a.year > b.year) {
return -1
}
return 0
})
})
.catch(error => {
console.log(error)
alert("An error has occurred.")
});
}
MapPane.vue
<template>
<div class="mapPane">
<l-map
:zoom="zoom"
:center="center"
:preferCanvas="true"
>
<l-control-scale
position="bottomleft"
:imperial="false"
:metric="true"
></l-control-scale>
<l-tile-layer
:name="tileProvider.name"
:visible="tileProvider.visible"
:url="tileProvider.url"
:attribution="tileProvider.attribution"
></l-tile-layer>
<Vue2LeafletMarkerCluster :options="clusterOptions" >
<LMarker v-for="anken in ankens" :key="anken.no" :lat-lng="makeLatLng(anken)" @click="onMarkerClick(anken.no)">
<LPopup :content="makeMarkerContent(anken)" ></LPopup>
</LMarker>
</Vue2LeafletMarkerCluster>
</l-map>
</div>
</template>
MapPane.vue
props: {
ankens:Array
},
MapPane.vue
<style scoped>
@import "~leaflet.markercluster/dist/MarkerCluster.css";
@import "~leaflet.markercluster/dist/MarkerCluster.Default.css";
</style>
Wrap LMarker in Vue2LeafletMarkerCluster and it will work. As a result, the close markers form a cluster (bundle) and are displayed together as one feature until it expands. As a result, problem (1) has also been resolved.
in conclusion
In the future, I would like to be able to sort and search the data list display, and display the animation being loaded. I arrived at Shizuoka PCDB when I was looking up the LIDAR data hotly, but it was an unexpected big derailment. Mr. Shizuoka, who uses such a large amount of valuable data as open data, is really Rock, and recently Hyogo Prefecture seems to have joined the ranks of Rocker. I feel that this project is a little different from the mainstream of open data utilization, but at least I think it has become a self-improvement. I would like to continue to set up an antenna in the open data area.
Recommended Posts