Introduction

Flood Warning Risk Distribution of the Japan Meteorological Agency Basin Rainfall Index There is data called .jma.go.jp/jma/kishou/know/bosai/ryuikishisu.html). This data represents the amount corresponding to the flow rate of the river, and the value is defined for each tertiary mesh code and river code, and is stored in BUFR4 format. There are various methods for decoding BUFR, but since the basin rainfall index uses a local table unique to the Japan Meteorological Agency, it cannot be read unless a local table is specified by any tool.

If you search for "basin rainfall index BUFR", you will find How to decrypt BUFR4 format, which is a Canadian BUFR. The method using the library libECBUFR is a hit. Of course, this is fine, but it's a bit cumbersome to install, and it's convenient to use Python, which is popular these days, so I used a Python package called PyBufrKit. investigated.

How to use

Installation

It seems that it cannot be installed with conda, so hit pip install pybufrkit.

Specifying a local table

Confirmation of the table used in the basin rainfall index BUFR

First, read Technical Information No. 446 regarding Distribution Materials, and then read the table used in the basin rainfall index BUFR. Check the parameters related to.

Or you can hit the pybufrkit command and you'll see an error. The parameters related to the table are as follows.

`info`


$ pybufrkit info (BUFR file)
(Abbreviation)
master_table_number = 0
originating_centre = 34
originating_subcentre = 0
master_table_version = 26
local_table_version = 1

Master table download

Download the JSON-formatted master table for PyBufrKit corresponding to the above parameters from PyBufrKit's GitHub. The files to be read are TableB.json and TableD.json in tables / {master_table_number} / 0_0 / {master_table_version} (there is also code_and_flag.json in the same directory, but this time it is missing But it worked). Therefore, in this case, https://github.com/ywangd/pybufrkit/tree/master/pybufrkit/tables/0/0_0/26 You can download the following tables of / tables / 0 / 0_0 / 26). The location of this downloaded tables directory is specified at runtime, so you can place it anywhere you like.

Let's check the contents of the downloaded JSON format table. First from Table B.

`0/0_0/26/TableB.json`


{"000001": ["TABLE A: ENTRY", "CCITT IA5", 0, 0, 24, "Character", 0, 3]}

This meaning can be easily understood by looking at the WMO Table B Specifications. The three behind are not used for CREX, so you can fill them with dummies when you create your own local table.

Next, let's take a look at Table D.

`0/0_0/26/TableD.json`


{
	"300002": ["", ["000002", "000003"]],
	"300003": ["(F, X, Y of descriptor to be added or defined)", ["000010", "000011", "000012"]]
}

The meaning of this can be understood by comparing it with WMO TableD specifications.

Creating a local table for JMA

Now that you know the format of the table, create a JMA local table in the following directory. tables/{master_table_number}/{originating_centre}_{originating_subcentre}/{local_table_version}/Table{B,D}.json Specifically, it becomes tables / 0 / 34_0 / 1 / Table {B, D} .json.

First, regarding Table B, the table called "Element Descriptor" in Technical Information No. 446 regarding Distribution Materials to see.

Therefore, the following files should be created.

`0/34_0/1/TableB.json`


{
    "008021": ["TIME SIGNIFICANCE", "CODE TABLE", 0, 0, 5, "CODE TABLE", 0, 2],
    "004001": ["YEAR", "YEAR", 0, 0, 12, "YEAR", 0, 4],
    "004002": ["MONTH", "MONTH", 0, 0, 4, "MONTH", 0, 2],
    "004003": ["DAY", "DAY", 0, 0, 6, "DAY", 0, 2],
    "004004": ["HOUR", "HOUR", 0, 0, 5, "HOUR", 0, 2],
    "004005": ["MINUTE", "MINUTE", 0, 0, 6, "MINUTE", 0, 2],
    "001210": ["RIVER CODE", "Numeric", 0, 80000000, 24, "Numeric", 0, 8],
    "005240": ["INDEX OF LATITUDE FOR 1ST ORDER MESH", "Numeric", 0, 0, 7, "Numeric", 0, 4],
    "006240": ["INDEX OF LONGITUDE FOR 1ST ORDER MESH", "Numeric", 0, 0, 7, "Numeric", 0, 4],
    "005241": ["INDEX OF LATITUDE FOR 2ND ORDER MESH", "Numeric", 0, 0, 4, "Numeric", 0, 2],
    "006241": ["INDEX OF LONGITUDE FOR 2ND ORDER MESH", "Numeric", 0, 0, 4, "Numeric", 0, 2],
    "005242": ["INDEX OF LATITUDE FOR 3RD ORDER MESH", "Numeric", 0, 0, 4, "Numeric", 0, 2],
    "006242": ["INDEX OF LONGITUDE FOR 3RD ORDER MESH", "Numeric", 0, 0, 4, "Numeric", 0, 2],
    "013212": ["RUNOFF INDEX (HIGH ACCURACY)", "Numeric", 1, 0, 12, "Numeric", 0, 4]
}

Similarly, Table D is called "aggregate descriptor" in Technical Information No. 446 regarding Distribution Materials. Look at the table.

Therefore, the following files should be created.

`0/34_0/1/TableD.json`


{
    "301011": ["", ["004001", "004002", "004003"]],
    "301012": ["", ["004004", "004005"]],
    "301200": ["", ["005240", "006240", "005241", "006241", "005242", "006242"]]
}

Command line usage

`help`


$ pybufrkit -h
usage: pybufrkit [-h] [-v] [--info | --debug] [-d DEFINITIONS_DIRECTORY]
                 [-t TABLES_ROOT_DIRECTORY]
                 command ...

--Specify the directory of the table created with -t --Reference: https://github.com/ywangd/pybufrkit/issues/5 --Display only file information - pybufrkit -t ./tables info ${bufrfile} --Decode to text format (quite redundant) - pybufrkit -t ./tables decode ${bufrfile} --Decode to JSON format (only values are stored) - pybufrkit -t ./tables decode -j ${bufrfile}

How to use as a Python module

Sample program to convert to JSON

`bufr2json.py`


import os
from pybufrkit.decoder import Decoder
from pybufrkit.renderer import FlatJsonRenderer


class BufrConverter:
    def __init__(self):
        self.TOPDIR = os.path.dirname(os.path.abspath(__file__))
        self.decoder = Decoder(tables_root_dir=f"{self.TOPDIR}/tables")
        self.renderer = FlatJsonRenderer()

    def convert(self, ini, region):
        filename = f"Z__C_RJTD_{ini}_MET_SEQ_Ggis1km_Proi_Aper10min_RJsuikei{region}_ANAL_bufr4.bin"
        filepath = f"{self.TOPDIR}/bufr/{filename}"
        with open(filepath, 'rb') as buf:
            bufr_message = self.decoder.process(buf.read())
        json = self.renderer.render(bufr_message)
        return json

A note of warnings and their meanings during work

If these warnings were not resolved, an error message would be output at the same time when outputting to JSON, which would be an obstacle. Note for reference.

WARNING: Fallback to default master table version 33 (26 not found) --If the master table tables / 0 / 0_0 / 26 specified by BUFR does not exist and the latest version of tables / 0 / 0_0 / 33 exists, it will fall back to that.
WARNING: Cannot find sub-centre 0 nor valid default. Local table not in use. --If the local table tables / 0 / 34_0 / 1 specified in BUFR is not found, the local table will not be used.

[PYTHON] Decode the basin rainfall index BUFR using PyBufrKit