[PYTHON] Play by converting the Japanese dialect corpus into a database (4) Decide the overall picture of the service

This is a serialized article. In the 4th time, the Laravel application created in the previous trial is ** once rebelled **, the whole picture of the service is decided, and the routing is created. It's a work memo for myself, and I don't think there are enough explanations, but please forgive me.

Functions you want to achieve

COJADS is currently available as a monitor version in Corpus search application "Chunagon", so it is functionally different from this. I want to make it.

List of utterances for each "discourse / speaker"

The "Chunagon" version mainly focuses on the search function, and does not provide text data as a group such as "discourse" or "speaker". In addition, as mentioned above, the raw data provided on the official website of COJADS is fully combined with "discourse" as the axis, which is inconvenient to handle. Therefore, it is of some value to provide the ability to separate this by entity so that it can be viewed on the site, for example, to view utterances by "speaker" as well as "discourse". It is considered.

Editing "Utterance"

You can operate the database from the site and add the function to edit the utterance without having to write SQL. It also logs [^ log]. I don't feel the need to do so, but I would like to implement it after studying the database.

[^ log]: I also made a log list page, but since there is nothing brand new in the explanation, I omitted it from this series of articles.

Mutual conversion between Excel and TextGrid

In COJADS, voice, dialect text, and standard language text are linked in units of utterances, but on the official website, only raw data in CSV format is distributed, and voice data and TextGrid format (voice analysis software Praat dedicated format) Text data is not distributed. The TextGrid file is directly linked to the audio, and I can't observe the correspondence well where the CSV and audio files are.

Also, even if TextGrid is distributed eventually, if you edit the CSV data by yourself, the correspondence will be broken and voice analysis will be difficult. Therefore, I would like to add a function to convert between Excel file and TextGrid file.

** I already have a Python script to convert between Excel and TextGrid **, so to make the most of this

--Upload the file --Convert with a Python script on the server --Download it

I will make it a mechanism.

Technology used

It's not as much as using AWS, so this time we'll use Heroku's free plan to implement the above features completely free of charge. As a web framework, use ** PHP Laravel ** as mentioned above. Also, since Heroku cannot use SQLite in the production environment [^ sqlite], we will use the free plan of ** Heroku PostgreSQL ** as the database.

[^ sqlite]: It seems that there are few services that can use SQLite in the production environment.

-Postgres --SQL Database Services | Heroku

Heroku is a service that you can start using very easily, but there are many parts that you can not use root privileges after building, especially in the free plan, and there are some difficult points for beginners, so there are some points. Will try to explain in Part 9.

Also, since it's a big deal, I will make it ** SPA (Single Page Application) **. There are several frameworks that implement SPA, but this time we will use Vue Router, which seems to have a low learning cost. The tutorial below was very helpful.

-Vue.js + Laravel to build a simple SPA tutorial: Overview | Qiita

Screen transition diagram

Count the required pages and create a simple screen transition diagram. I will draw it in several parts so that it will not be unpleasant to display on Qiita.

It's not elegant, but I used draw.io because it's iconic, easy to understand, and the drawing cost is low.

Let's do the utterance list and editing function for each "discourse" as follows. The "Discourse" table contains a great deal of information (file symbols, data names, dates of recording, recording locations, recording staff, editors, topics, discourse genres), so all this is one. It gets in the way when displayed on the screen. Therefore, the "Discourse List" page should not display elements that do not affect the nature of the data (recorders, editors, etc.) so that they can be confirmed only on the "Utterance Details" page. Also, after updating on the "Edit utterance" page, move to the "Details of utterance" page.

The utterance list for each "speaker" is boring to simply summarize the utterances, so let's chew on a simple character string processing and present it by sentence instead of by utterance.

The file conversion screen should be uploaded, converted, downloaded, and deleted for the time being, so prepare a simple one.

Project preparation

Let's create it. First, let's create a Laravel project. The things to do are basically the same as the above reference site, have composer create a project in an appropriate folder, and introduce the necessary packages etc. with the help of Laravel (php artisan) and npm. I will continue.

composer create-project laravel/laravel --prefer-dist cojads
composer require laravel/ui
php artisan ui vue
npm install
npm install --save vue-router

From now on, I will make the content of the project, but I will follow the procedure of starting from the river (routing and template) and creating the content (components and controllers) later.

SPA template creation

This time we will create a SPA, so we will create the only page ʻapp.blade.phpas a river (template). In the Laravel project, the routing for normal access as a website is specified inweb.php, so first edit this and route any path to ʻapp.blade.php. However, this service will need to access / storage directly later, so do not route anything under / storage.

`routes/web.php`


<?php
use Illuminate\Support\Facades\Route;
Route::get('/{any}', function() {
    return view('app');
})->where('any', '^(?!storage).*');

Then create a SPA river ʻapp.blade.php. The Vue Router process loads the header at the position, the footer at theposition, and the content of each page at the` position. ..

In order to create a fixed header with bootstrap, there are appropriate margins above and below <rooter-view> (dirty).

`html:resources/views/app.blade.php`


<!doctype html>
<html lang="{{ str_replace('_', '-', app()->getLocale()) }}">
<head>
    <base href="/" />
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <meta name="robots" content="noindex">
    <meta name="csrf-token" content="{{ csrf_token() }}">
    <title>{{ config('app.name', 'COJADS App') }}</title>
    <link href="{{ mix('/css/app.css') }}" rel="stylesheet">
</head>
<body>

<div id="app">
    <header-component></header-component>
    <div class="px-5 py-5"><div class="py-3">
        <router-view></router-view>
    </div></div>
    <footer-component></footer-component>
</div>

<script src="{{ mix('/js/app.js') }}" defer></script>

</body>
</html>

Component routing

It then routes the components that will be inserted into the template. The registration of components and their routing is managed by ʻapp.js`, so edit it here. In the following, the elements of the header, footer, cover page, and discourse list page are read and appropriate routing is performed.

`resources/js/app.js`


import VueRouter from "vue-router";

// register components
import HeaderComponent from "./components/HeaderComponent";
import FooterComponent from "./components/FooterComponent";
import HomeComponent from "./components/HomeComponent";
import DiscourseIndexComponent from "./components/DiscourseIndexComponent";

require("./bootstrap");
window.Vue = require("vue");
Vue.use(VueRouter);

// routing
const router = new VueRouter({
    mode: "history",
    routes: [
        {
            //Load HomeComponent when accessing root
            path: "/",
            name: "home",
            component: HomeComponent
        },
        {
            // /Load DiscourseIndexComponent when accessing discourse
            path: "/discourse",
            name: "discourse.index",
            component: DiscourseIndexComponent
        }
    ]
});

//Define a custom element for the header / footer and load the component
Vue.component("header-component", HeaderComponent);
Vue.component("footer-component", FooterComponent);

const app = new Vue({
    el: "#app",
    router
});

Creating each component

Once you have a routing concept, you can create the content of the page. Create the four components (.vue files) you decided to use in ʻapp.js earlier under / resources / js / components (discourse` will be next time).

`Create the following files`


+ resources/js/components/HeaderComponent.vue
+ resources/js/components/FooterComponent.vue
+ resources/js/components/HomeComponent.vue
+ resources/js/components/DiscourseIndexComponent.vue

header

Create a type of header that is fixed at the top of the page with bootstrap. As a prelude, let's put a link to each component with <router-link> in the header. However, I haven't provided a link yet, so if you want to compile at this stage, you need to comment it out appropriately.

-API Reference # \ | Vue Router

`resources/js/components/HeaderComponent.vue`


<template>
    <header class="navbar fixed-top navbar-dark bg-dark">
        <span class="navbar-brand mb-0 h1">COJADS APP</span>
        <div>
            <span>
                <router-link v-bind:to="{ name: 'home' }">
                    <button class="btn btn-success btn-sm">Top Page</button>
                </router-link>
                <router-link v-bind:to="{ name: 'discourse.index' }">
                    <button class="btn btn-success btn-sm">Discourse List</button>
                </router-link>
                <router-link v-bind:to="{ name: 'speaker.index' }">
                    <button class="btn btn-success btn-sm">Speaker List</button>
                </router-link>
                <router-link v-bind:to="{ name: 'convert' }">
                    <button class="btn btn-success btn-sm">Toolkits</button>
                </router-link>
            </span>
        </div>
    </header>
</template>

<script>
export default {};
</script>

footer

Make the footer appropriately.

`resources/js/components/FooterComponent.vue`


<template>
    <footer class="navbar fixed-bottom navbar-dark bg-dark">
        <span class="navbar-brand mb-0 h1">(c) 2020 @a_eau_</span>
    </footer>
</template>

<script>
export default {};
</script>

Home Screen

I don't have to write anything in particular, so I will make only the minimum.

`resources/js/components/HomeComponent.vue`


<template>
    <div>
        Welcome to COJADS App!
    </div>
</template>

<script>
export default {};
</script>

Compile / Check on local server

After writing the above code, compile it once and check it. If you execute the following command, it will take in all the necessary things and compile it nicely.

-Compile Asset (Mix) 7.x Laravel

`cmd`


npm run dev or npm run production

After compiling, start the local server with the following command of Laravel and try to access localhost: 8000. If you get the expected page without a 404 or 500 error, you're good to go.

`cmd`


php artisan serve

next time

It's time to switch to PostgreSQL after learning that SQLite isn't available on Heroku.

--Fifth: Playing by converting the Japanese dialect corpus into a database (5) Database migration and model creation