docker run --runtime=nvidia -it tensorflow/tensorflow:1.15.2-gpu-py3 bash

When building an environment quickly with Conda

conda create -n gpt-2 python=3.6
conda activate gpt-2
pip install tensorflow-gpu==1.15

>>> gpt2.generate(sess)
Cells in the body of a whale are often used as a means to induce sleep in some whales. But in the case of a whale, this particular type of whale is known as a cusps whale because it is usually about the size of a human hair.

The, or humpback whale, is one of the largest cusps whales in the world. It is an extremely large, highly muscled, and highly territorial mammal, with a very large mouth and, in some sections, white, skinned head.

...

Northeast Regional, which runs through the corridor 188 (Union Station)(Washington D. C .)Depart Pennsylvania Station(New York)Departed from Philadelphia's 30th Street Station. The train is an ACS that manufactured 7 passenger cars a year ago.-Type 64 electric locomotive( No . 601 )Was towed.

Approximately 11 minutes later, the train is running on the double-track main line southeast of, at 4 degrees (approximately 440 m in radius) near the intersection of Frankford Avenue and Wheatseef Lane in the Port Richmond district. ) Entered the left curve.
2015 Amtrak derailment accident

# pip install tensorflow==
ERROR: Could not find a version that satisfies the requirement tensorflow== (from versions: 2.2.0rc1, 2.2.0rc2)
ERROR: No matching distribution found for tensorflow==

When I install the latest version for the time being and proceed, the following error occurs.

# python3 src/generate_unconditional_samples.py | tee /tmp/samples
Traceback (most recent call last):
  File "src/generate_unconditional_samples.py", line 9, in <module>
    import model, sample, encoder
  File "/target/src/model.py", line 3, in <module>
    from tensorflow.contrib.training import HParams
ModuleNotFoundError: No module named 'tensorflow.contrib'

Upon examination, 'tensorflow.contrib' seems to be [obsolete] in tensorflow 2.x (https://github.com/tensorflow/tensorflow/issues/31350#issuecomment-518749548) ... There is no help for it, so switch to the installation from dockerfile. Here, the image of tensorflow / tensorflow: 1.12.0-py3 is used. If you drop all the models, it will be heavy, so modify it so that only the lightest model is dropped.

Dockerfile.cpu

FROM tensorflow/tensorflow:1.12.0-py3

ENV LANG=C.UTF-8
RUN mkdir /gpt-2
WORKDIR /gpt-2
ADD . /gpt-2
RUN pip3 install -r requirements.txt
RUN python3 download_model.py 124M

If you can edit it, it will be executed on the CPU for the time being, so it looks like the following ...

docker build --tag gpt-2 -f Dockerfile.cpu .
docker run -it gpt-2 bash

export PYTHONIOENCODING=UTF-8
python3 src/generate_unconditional_samples.py | tee /tmp/samples

Then I got the error ʻAttributeError: module'tensorflow' has no attribute'sort'. Upon examination, it seems that tensorflow 1.12.0 needs to be modified to import tf.contrib.framework.sort... It seems to work if you usetensorflow 1.14.0`, so this time I would like to modify the dockerfile side.

Dockerfile.cpu

#Because it's a big deal, 1.I made it the latest version of x series
FROM tensorflow/tensorflow:1.15.2-py3

ENV LANG=C.UTF-8
RUN mkdir /gpt-2
WORKDIR /gpt-2
ADD . /gpt-2
RUN pip3 install -r requirements.txt
RUN python3 download_model.py 124M

So, if you take a second look and run it, you will get the following lie article output!

python3 src/generate_unconditional_samples.py | tee /tmp/samples
======================================== SAMPLE 1 ========================================
 — President Donald Trump tweeted on Saturday that he would not do it again in the 2017 budget.

"Of course, and I bet WA will accept my debt — but if a bad story develops, they'll tell me as long as I am cutting deduction for health care," Trump tweeted on December 19.

If a bad story develops, they'll tell me as long as I am reducing deduction for health care. — President Donald Trump (@realDonaldTrump) December 19, 2017  

The first budget request "is building around a debt epidemic for $3.5 trillion," according to CNN. The problem, it turns out, is that Trump would work with 
Republicans to pass a debt-ceiling increase, despite claims that the written framework can't be passed.

The budget would create $11.1 trillion in government debt, according to PPP , Russia, and China – which have agreed on a plan to get rid of regulations on corporate taxes as part of a five-year plan which includes massive cuts to subsidies for growth to deal with the looming financial crisis.

Today's budget contradicts Cliff's upcoming budget agreement, which to...

Certainly, even the smallest model can make an article that feels like ... amazing. Now that I've finally moved the sample, I'd like to learn it next time.

Learning using GPT-2

To be honest, there are too few places to explain, so it's quite difficult ... Maybe it's because it's not abused ...

Use of Japanese corpus

Issue 104, 114 says the corpus It is said that it is Byte_pair_encoding. If you want to make your own model, refer to This person's way and [SentencePiece](https: / (/github.com/google/sentencepiece) looks good.

This time, we will borrow BERT model learned from Mr. Possible's Japanese Wikipedia. Download Trained Model ...