The other day, I wrote an article "Getting Flux results of Spring Web Flux from JS with Fetch API", but using this method, For example, when you want to display a large amount of data in the browser, if the server can process only the first few tens of data, display the first few tens of data in the browser first, and then add the processed data in the background. To go. I thought it would be possible to do something like ** easily **, so I tried it.

If the user requests a large amount of data, there is an option to implement it by paging, There may be such an option. It is an article about. We do not recommend this method.

The one I tried here is uploaded to GitHub.

Processing content

Request 1 million data from the server and display the results in the browser grid.

Technical elements used

--Server side

Java11
Spring WebFlux
H2 Database
Database & jdbc. Since it is troublesome to set up a DB server, I made it a file DB with H2.
uroboroSQL
DB access library. Our OSS. I could have used anything, but I'm used to this so I'll use it. --Client side
Vue.js
I could have used anything, but I like Vue and I like it because it's easy to build with Vue CLI.
Cheetah Grid
Grid library. Our OSS that I made. I intend to use this because I intended to display 1 million items.

Comparison and video

I will show you the comparison result first.

Asynchronous request using Spring WebFlux

The time to the first drawing was 2,056ms. I thought it would be a little earlier, but it seems that it takes time for the first one to be returned after issuing the SQL query in H2 Database. (Well, I don't think H2 is for production use, so it can't be helped.) The video has been cut, but after that, it takes about 40 seconds to acquire the chorochoro data and store 1 million on the client side.

Ordinary sync request

The time to draw was 5,827ms. Since I bring them all at once, it is the same 5,827ms in terms of the time it takes to store 1 million items on the client side. I thought it would be slower, but I didn't do anything to build each record, and there wasn't much information on one record, so was it surprisingly fast?

Implementation

I will also write about the implementation below.

Controller (server side)

The implementation is as follows.

	@Autowired
	private SqlConfig uroboroSQLConfig;

	@Autowired
	@Qualifier("jdbcScheduler")
	private Scheduler jdbcScheduler;

	@GetMapping
	public Flux<Person> all() {
		return Flux.<Person> create(sink -> {
			try (var agent = uroboroSQLConfig.agent()) {
				//Select the table and generate a Stream mapped to the Person class
				agent.queryWith("SELECT * FROM PERSON ORDER BY ID")
						.stream(Person.class)
						//Exit if canceled
						.dropWhile(p -> sink.isCancelled())
						//Pass Entity to FluxSink
						.forEach(sink::next);

				if (!sink.isCancelled()) {
					//Notify the end
					sink.complete();
				}
			}
		}).subscribeOn(jdbcScheduler);
	}

ʻUroboroSQLConfig` is required for DB access of uroboroSQL. It has nothing to do with the main subject, so don't worry about it, just think it's something for DB access and read it in the air.

The jdbcScheduler is aScheduler instance of Reactor that is required for parallelization.

Define Scheduler with the appropriate number of threads,

		}).subscribeOn(jdbcScheduler);

By passing, the process of DB access and the process of returning a request are parallelized.

The rest is cancellation processing, Basically, just SELECT from the DB and pass the result withsink.next (...).

Request (client side)

Fetch API by the method described in Getting Flux results of Spring Web Flux from JS with Fetch API Make a request to the server using .org / ja / docs / Web / API / Fetch_API).

Display on grid (client side)

The implementation is as follows.

      const records = [];
      const grid = /*Instance of Cheetah Grid*/;
      let buffer = [];
      //...
      streamJsonForVue(this, "/api/persons", {}, rec => {
        buffer.push(rec);
        if (
          buffer.length >= 10000 ||
          (records.length < 10000 && buffer.length >= 1000) ||
          (records.length < 1000 && buffer.length >= 100)
        ) {
          records.push(...buffer);
          grid.records = records;
          buffer = [];
          //...
        }
      }).then(() => {
        records.push(...buffer);
        grid.records = records;
        //...
      });

The source posted on GitHub is around hereis.

The grid variable contains an instance of Cheetah Grid, and if you give an array to the records property, the given data will be displayed on the screen. It shows.

In the streamJsonForVue function, I wrote in" Getting Flux results of Spring Web Flux from JS with Fetch API "Fetch API is being processed. Plus, I'm doing things like interrupting Fetch when the screen is discarded, but I'll omit the explanation because it has nothing to do with the main subject.

The rec variable returned from the streamJsonForVue callback is the one-record data passed by sink.next (...) in the Controller. If you pass it to grid one by one, redrawing will run and it will be difficult for the user to operate the screen, so store it in the buffer array, and when it accumulates to some extent (first 100 cases), reflect it in grid I am doing it.

Implementation of synchronization request

The source posted on GitHub is [around here](https://github.com/ota-meshi/spring-webflux-x-cheetah-grid-example/blob/master/src/main/java/example/spring /webflux/controller/PersonController.java#L48) and [around here](https://github.com/ota-meshi/spring-webflux-x-cheetah-grid-example/blob/master/src/components/SampleCheetahGrid .vue # L131). I made it for comparison only, but it has nothing to do with the main subject, so I will omit the explanation.

Impressions implemented

What I thought about implementing it was that it was ** very easy to implement **. To my knowledge, it was difficult to send HTTP response to Stream asynchronously, but I was surprised because it was very easy to do with Spring WebFlux.

result

As I wrote earlier, in this sample, the time required for initial drawing was as follows.

--Asynchronous request using Spring WebFlux 2,056ms --Ordinary sync request 5,827ms

It was a pity that this sample was not as effective as I expected, but it may be useful if you really want to speed up the initial drawing (?).

I thought that the initial drawing would be faster if I used Spring WebFlux to get 1 million data from the server and display it on the browser, so I tried it