[PYTHON] [LSTM text generation] Use ml5js

Day 24 of Javascript Advent Calendar 2019

What is LSTM?

The first character is given to the model that learned the sentence data, and the next character is guessed.

Library used

ml5js ml5 It uses the tensorflow model and makes it easy to handle with js.

tensorflow It is used for model generation. tensorflow

Advance preparation

What to install

Python --Needed to run tensorflow
Anaconda --Python distribution. A collection of libraries used in machine learning and scientific calculations. Python itself is also installed.
pip --Used to install Python packages (TensorFlow is included with pip)
nodejs --ml5 library Used to move the front end side

I installed Python with Anaconda and TensorFlow with pip.

environment

OS:windows 10 (It's a gaming PC. I thought it would take a certain amount of load.)

Training set

Download here training-charRNN

git clone https://github.com/ml5js/training-charRNN.git

Prepare the data

This is the most difficult place both later and earlier, but I will prepare appropriate text data so that I can move it for the time being.

Preparing Japanese data is difficult to separate phrases, so let's use English data. I will summarize Japanese at a later date.

File name: input.txt Location: training-lstm-master / [any_holder_name] /

Enter the text in input.txt and save it.

reference data

Free English novels and stories are available. If you want to use it as a trial, I think you can just prepare the original data from here. Project Gutenberg

Train

--Launch Anaconda Prompt --Move to the directory of the training set.

cd training-lstm-master

--Train the created data

python train.py --data_path=./[any_holder_name]/input.txt

After this training starts and ends, a file is generated in models / input /.

Generated files

embedding
manifest.json
rnnlm_multi_rnn_cell_cell_0_basic_lstm_cell_bias
rnnlm_multi_rnn_cell_cell_0_basic_lstm_cell_kernel
rnnlm_multi_rnn_cell_cell_1_basic_lstm_cell_bias
rnnlm_multi_rnn_cell_cell_1_basic_lstm_cell_kernel
rnnlm_softmax_b
rnnlm_softmax_w
Variable
vocab.json

We will use this file group in ml5-examples which will be explained later.

If an error occurs

I had the latest version of tensorflow, but I was angry. (I'm sorry for the error content at that time, but it has become a thousand winds.)

I had no choice but to put ** tensorflow 1.15.0 **.

I have to rewrite it to a 1-series function, and the following of train.py is modified.

# hide logs
tf.logging.set_verbosity(tf.logging.ERROR)

↓

# hide logs
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)

I haven't talked about the front end yet, so I'm worried that this is good for Advent ...

front end

I finally got to the story of javascript. Now, let's use the model created earlier.

Move files

Download here ml5-examples

git clone https://github.com/ml5js/ml5-examples.git

I'm using the nodejs package, so it's the usual one

npm install

Move files

Move the files under models / input / generated earlier.

Destination https://github.com/ml5js/ml5-examples/tree/release/p5js/CharRNN/CharRNN_Text/models/woolf

It is recommended that the input folder be a folder name that matches the text.

Example) Text about cats ml5-examples-master/p5js/CharRNN/CharRNN_Text/models/cat

Text about fortune-telling ml5-examples-master/p5js/CharRNN/CharRNN_Text/models/horoscope

Specify model

Since models is specified in sketch.js, replace "woolf" with the folder name created under models earlier.

charRNN = ml5.charRNN('./models/woolf/', modelReady);

https://github.com/ml5js/ml5-examples/blob/release/p5js/CharRNN/CharRNN_Text/sketch.js#L24

Launch local server

python 3 series

python -m http.server

python 2 series

python -m SimpleHTTPServer

If you open http: // localhost: 8000 / you will see the directory.

Use CharRNN Text

Click pl5js

FireShot Capture 017 - Directory listing for _ - localhost.png

Click CharRNN FireShot Capture 018 - Directory listing for p5js - localhost.png

Click CharRNN_Text FireShot Capture 019 - Directory listing for p5js_CharRNN - localhost.png

You will reach the Generator screen.

seed text: The first character to give length: number of characters you want to generate temperature: Weight / depth

FireShot Capture 022 - LSTM Text Generation Example - localhost.png

seed text:happy length：100 Sentence generated at temperature: 0.5

happy and the other grown herself, 'I was than the bottle my to little sing how the poor comly up and gut

FireShot Capture 023 - 翻訳 - Google 検索 - www.google.com.png

seed text:happy length：100 Sentence generated by temperature: 1

happys! All spomes wife a 'How finE it?' said 'Hares" should neven backed as much had right gaim--'

FireShot Capture 026 - 翻訳 - Google 検索 - www.google.com.png

The text is incoherent.

What sentence did I use as data? You can find hints in the latter generated text.

Hint "Rabbit".

Answer: Alice's Adventures in Wonderland

The latter sentence has a temperature of 1, so it should be more like Alice's Adventures in Wonderland than the former (temperature: 0.5).

Certainly, I feel that the "rabbit" and "rights" have that kind of feeling.

I want to find out more about ml5.CharRNN.

If you use mecab and prepare Japanese sentence data, you may be able to use Japanese sentences! I heard that, so I will take time to try it. It's not completely about javascript lol

Reference article

-[4_1_2: LSTM training (creating "Alice in Wonderland" model) ml5.js JavaScript](https://himco.jp/2018/12/31/4_1_2%EF%BC%9Alstm%E3%81 % AE% E8% A8% 93% E7% B7% B4% E3% 80% 8C% E4% B8% 8D% E6% 80% 9D% E8% AD% B0% E3% 81% AE% E5% 9B% BD % E3% 81% AE% E3% 82% A2% E3% 83% AA% E3% 82% B9% E3% 80% 8D% E3% 83% A2% E3% 83% 87% E3% 83% AB% E3 % 81% AE% E4% BD% 9C% E6% 88% 90 /) -[4_1_1: LSTM text generation sample ml5.js JavaScript](https://himco.jp/2018/12/30/4_1_1%EF%BC%9Alstm-%E6%96%87%E7%AB%A0%E7 % 94% 9F% E6% 88% 90% E3% 82% B5% E3% 83% B3% E3% 83% 97% E3% 83% AB /)