Display GIS data in ElasticSearch in QGIS via GeoServer

Overview

I tried to distribute GeoJSON data in ElasticSearch with GeoServer and display it with QGIS.

We will set up the following apps using Docker.

GeoServer 2.16.2
ElasticSearch 7.9.3
Kibana 7.9.3

Docker setup

If Docker is not installed, first install it according to the Official Site.

Launch GeoServer, ElasticSearch, Kibana

starting method

First, the series of files covered in this article are located in JinIgarashi / docker-geoserver-elasticgeo.

If you do the following, a series of applications such as GeoServer will be launched.

git clone [email protected]:JinIgarashi/docker-geoserver-elasticgeo.git
cd docker-geoserver-elasticgeo
docker-compose up

About apps launched with Docker

When started with docker-compose, GeoServer will start on port 8600, ElasticSearch on port 9200, and Kibana on port 5601.

GeoServer settings are the default of kartoza / docker-geoserver It is diverted from .env and docker-compose.yml of, and the setting of Elastic Search is the same as the official one, but the number of nodes is only one.

However, since it is necessary to put an extension called ElasticGeo in GeoServer, I have created my own Dockerfile as follows.

FROM kartoza/geoserver:2.16.2

RUN wget https://github.com/ngageoint/elasticgeo/releases/download/2.16.0-RC1/elasticgeo-2.16.0.zip -O elasticgeo.zip
RUN unzip -o elasticgeo.zip -d /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/

EXPOSE 8080

Simply place the ElasticGeo jar file in the GeoServer WEB-INF folder and the installation is complete. Note that ElasticGeo does not work with the latest GeoServer 2.18 at this time, so use 2.16.2.

Check ElasticSearch

If you execute the following in the terminal and the result is returned, ElasticSearch is successfully installed.

$ curl http://localhost:9200
{
  "name" : "es01",
  "cluster_name" : "es-docker-cluster",
  "cluster_uuid" : "W_ho6j3NSZWQvuL7CfNnXg",
  "version" : {
    "number" : "7.9.3",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "c4138e51121ef06a6404866cddc601906fe5c868",
    "build_date" : "2020-10-16T10:36:16.141335Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Import GeoJSON in Kibana

Open Kibana in your browser

Go to http: // localhost: 5601. It is OK if a screen like the one below appears.

Upload GeoJSON to ElasticSearch

Here, I will upload more than 50,000 point data of the water meter of the Rwanda Waterworks Corporation, which I work with as a volunteer.

Open the menu on the top left and open Map.

Click Add layer.

Click Upload GeoJSON.

If you drag and drop your GeoJSON into the space of Select a file to upload, it will be loaded automatically, so click Import file continuously to start indexing.

When it changes to the Add layer button, click it continuously. Enter the appropriate information in Layer Settings and save. I just named it rw_connections and left the rest as default

Then I was able to display it in Kibana with the following feeling.

Kibana's GeoJSON maximum size is up to 50MB

If you exceed the maximum size limit of 50MB in Kibana, you can import it by using the command below with ogr2ogr. However, if you import with ogr2ogr, you have to create the index pattern yourself with Kibana.

ogr2ogr -progress -lco BULK_SIZE=5000000 -f "Elasticsearch" http://localhost:9200 connections.geojson -skipfailures

GeoServer settings

Now that the data is ready, let's set up GeoServer.

Go to http: // localhost: 8600 / geoserver. A familiar screen will appear.

ID: admin
Password: myawesomegeoserver

Create a workspace.

Create a store. Since ElasticGeo is set, there is an option called Elasticsearch in Vector Data Sources, so select it.

Enter the index information you uploaded in Kibana.

--Data Source Name: Give it any name --elasticsearch_host: Enter the Docker container name es01 --elasticsearch_port: Put 9200 --index_name: connections

Leave the other defaults and click Save.

Next, add an ElasticSearch layer. If the store is set up properly, the following screen will be displayed. Click Publish.

It will be a layer edit screen, but this time we will create a layer after setting only BBOX with all the defaults.

You have a layer.

Let's take a look at the GeoServer layer created by Layer Previw.

Click OpenLayers and a map will appear.

View data in QGIS

Now that GeoServer can deliver ElasticSearch data, let's view it in QGIS.

This time I will use the latest QGIS 3.16 at the moment.

After launching QGIS, right-click on WFS / OGC API in the browser panel and click New Connection.

--Name is elasticsearch --URL to http: // localhost: 8600 / geoserver / elasticsearch / wfs --Version to 1.0.0 --The maximum number of features is 10000 Fill in and click OK.

If you don't know the WFS URL, open the URL below in your browser and you can see the URL you specify at OnlineResource. http://localhost:8600/geoserver/elasticsearch/ows?service=WFS&version=1.0.0&request=getcapabilities

<Service>
<Name>WFS</Name>
<Title/>
<Abstract/>
<Keywords/>
<OnlineResource>http://localhost:8600/geoserver/elasticsearch/wfs</OnlineResource>

The connections layer is displayed in the browser panel. Double-click connections to add a layer.

I was able to display the WFS layer properly in QGIS 3.16. In Kibana, you cannot see the displayed feature information in pop-ups, but if you display it in QGIS via GeoServer, you can see the attribute information properly.

Summary

--GIS data input to ElasticSearch can be distributed by WFS / WMS by installing ElasticGeo extension in GeoServer. --Can be displayed in QGIS using the WFS / WMS distribution method --In addition to QGIS, Leaflet, OpenLayers, Mapbox GL JS, etc. can handle ElasticSearch data via GeoServer (not covered by this article).

I felt that the handling of GIS data in ElasticSearch became more familiar by using GeoServer.