I want to display the progress in Python!

Overview

When you execute a process that takes a long time, if there is no response, do you feel anxious about "how far the process is progressing" or "is it working?" There is, right? Yes, there is (three-row conjugation). So I will make a note of how to display the progress of loop processing.

↓ ↓ ↓ By the way, it looks like this ↓ ↓ ↓ If you find it useful, I would be grateful if you could comment on LGTM stock.

table of contents

-[Use package tqdm](#Use package tqdm) -[Try to make it yourself](# Try to make it yourself) -[Escape Sequence](# Escape Sequence) -[Try to make a progress bar](# Try to make a progress bar) -[Development: ANSI escape code](#Development ansi escape code)

Use of package tqdm

First of all, I would like to introduce tqdm, the royal road of progress display (I think it is selfish). It's very easy to use, just import and embed it in your loop like this:

tqdm_test.py

%!pip install tqdm
import tqdm
import numpy as np


for i in tqdm.tqdm(range(int(1e7))):
    np.pi*np.pi

Now you should see something like this: It's convenient ~ Since you are passing an iterable (loopable) object to tqdm.tqdm (), you can also pass lists, dictionaries, strings, and so on. However, as a caveat, if there is a standard output print function etc. in the loop processing, it will be terrible.

tqdm_test.py

%!pip install tqdm
import tqdm
import numpy as np


for i in tqdm.tqdm(range(int(1e7))):
    np.pi*np.pi
    if i % 1e6 == 0:
        print(i)

By the same theory, it would be terrible to show progress with the tqdm.tqdm function for nesting loops.

tqdm_test.py

import tqdm
for t in tqdm.tqdm(range(10)):
    for i in tqdm.tqdm(range(int(1e6))):
        np.pi*np.pi

This kind of thing is very rarely inconvenient, isn't it? Do you all have such an experience? There is, right? Yes, there is (three-row conjugation). So let's do something about it ourselves.

Try to make it yourself

At least in Python, it seems that the standard output string leaves control of the program at the time of line break **. Conversely, it means that it can be operated programmatically even after standard output as long as there is no line break **.

progress.py

for i in range(int(1e7)):
    if i % 1e4 == 0:
        print("\r", i, end="")
    np.pi*np.pi

These \ r andend = ""allow the program to continue to take control after standard output. end =" " is an option to specify the character string to be added to the end of the character string output by the print function, and \ n (line feed) is specified by default. Therefore, if you change it to add an empty string by setting end =" " (although it is strange to say), line breaks will not occur, so you can play with the standard output programmatically. As it appeared here, \ r and \ n starting with a backslash are called ** escape sequences **, and are a group of strings that enable special control that you want to include in the string. I will.

Escape sequence

As mentioned above, an escape sequence is a group of character strings used when you want to give special control to a character string. The typical ones are as shown in the table below.

In terms of processing, the backslash indicates the beginning of the escape sequence description, and it is converted to the control statement or character code corresponding to the character string that follows it. Therefore, it is also used when outputting the character string used in the control statement of the character string itself such as double quotation " .

test_esc.py

print("\"")
print("\'")
print("\\")

It's possible to use this feature to read a string in Unicode, but that's probably not the case.

test_esc.py

# Hello world!
print("\x48\x65\x6c\x6c\x6f\x20\x77\x6f\x72\x6c\x64\x21")

Try to make a progress bar

Let's create our own progress bar using the escape sequence \ r.

progress.py

import time
for i in range(50):
    print("\r[" + "#"*i + "]", end="")
    time.sleep(0.1)

For the time being, we have created something that can be called a progress bar! By the way, I don't know why, but the escape sequence doesn't work when I use the format function. It doesn't taste good as it is, so let's try more.

progress.py

import time
epoch = 50
for i in range(epoch):
    bar = "="*i + (">" if i < epoch-1 else "=") + " "*(epoch-i-1)
    print("\r[" + bar + "]", "{}/{}".format(i+1, epoch), end="")
    time.sleep(0.1)

As an explanation, first write the main character string of the progress bar in the variable bar. In the code, spaces are filled in to fix the length of the progress bar, and the beginning is an arrow.

Besides, you can do anything because you only have to define the nesting that you couldn't do with tqdm.

progress.py

import time


t_epoch = 10
i_epoch = 50
lap_time = -1
start_time = time.time()
for t in range(t_epoch):
    t_per = int((t+1)/t_epoch*i_epoch)
    for i in range(i_epoch):
        i_per = int((i+1)/i_epoch*i_epoch)
        if i_per <= t_per:
            bar = ("progress:[" + "X"*i_per
                                + "\\"*(t_per-i_per)
                                + " "*(i_epoch-t_per) + "]")
        else:
            bar = ("progress:[" + "X"*t_per
                                + "/"*(i_per-t_per)
                                + " "*(i_epoch-i_per) + "]")
        time.sleep(1e-2)
        elapsed_time = time.time() - start_time
        print("\r" + bar, "{}s/{}s".format(
                int(elapsed_time),
                int(lap_time*t_epoch) if lap_time > 0 
           else int(elapsed_time*(i_epoch/(i+1))*t_epoch)),
              end="")
    lap_time = (time.time() - start_time)/(t+1)

It suddenly became complicated ... I will explain. t_per and i_per calculate and hold the number of characters required to display the progress in the t loop and i loop as a character string. bar is the progress bar itself

--The t loop uses the string\to indicate progress. --The i loop uses the string/to indicate progress. --The overlapping part is the character string X to indicate the progress.

It is programmed to be like this. The backslash is the start character of the escape sequence, so you need to use " \\ ". lap_time holds the time taken for one loop of i. I take the average to get a more accurate value. elapsed_time is the elapsed time to the present.

progress.py

                int(lap_time*t_epoch) if lap_time > 0 
           else int(elapsed_time*(i_epoch/(i+1))*t_epoch))

However, if lap_time is calculated, it is displayed, and if it is not (that is, when the loop of i is entered for the first time), the lap time is estimated from the elapsed time and displayed.

This is just an example, so please think about it.

Evolution: ANSI escape code

So far, I said that "the standard output cannot be overwritten if there is a line break", but in fact, it can be operated even if there is a line break. The method is ** ANSI escape code **.

progress.py

import time
epoch = 25

print("　"*(epoch-10) + "彡 ノ ノ ノ Hami ⌒ Mi")
print("　"*(epoch-11) + " (´ ・ ω ・ ｀)ω ・ ｀)now! Shoot with Ora!")
print("　"*(epoch-13) + "⊂ ∩ ∩ tsu)")
print("　"*(epoch-10) + "/　　　〈　　 〈")
print("　"*(epoch-11) + " ( /⌒ ｀ J ⌒'")
print("\n\n")
print("　"*(epoch-1)  + "No")
print("　"*(epoch-4)  + "彡 ノ")
print("　"*(epoch-6)  + "No")
print("　"*(epoch-8)  + "Nono Minono")
print()
print("　"*(epoch-11) + "(´；ω；｀)ω^｀)Shit ah ah ah! !! !!")
print("　"*(epoch-13) + "⊂ ∩ ∩ tsu)")
print("　"*(epoch-10) + "/　　　〈　　 〈")
print("　"*(epoch-11) + " ( /⌒ ｀ J ⌒'")
print("\033[6A")
for i in range(epoch):
    bar = "弌"*i + "⊃" + "　"*(epoch-i-1)
    print("\to r" + bar + "]", "{}/{}".format(i+1, epoch), end="")
    time.sleep(0.1)
print()
print("\033[5B")

Please pass through if the way to hide personal information is complicated lol Do you know It is a parody AA of the famous scene of Dragon Ball, the game against Raditz. (I found something, so I used it) The code to display the AA part is pretty much what you see. However, note that all AAs have been output in advance. And

progress.py

print("\033[6A")

Pre-opened by

progress.py

print("　"*(epoch-8)  + "Nono Minono")
print()
print("　"*(epoch-11) + "(´；ω；｀)ω^｀)Shit ah ah ah! !! !!")

I moved to the empty line part of and displayed the Manuki Kosatsu method in the same way as before. Finally

progress.py

print()
print("\033[5B")

With, the Manuki Kosatsu method is broken and moved to the last line of standard output.

Now, the mysterious code is coming out, this is the ** ANSI escape code **. In Python, there are ANSI escape codes in the following table.

If you have any questions, let's make only the first four available. By using this ANSI code, you can operate with a high degree of freedom as before.

By the way, this ANSI code seems to be usable only when using a console such as a terminal, so let's put up with the single-line progress bar at \ r in jupyter notebook etc.

in conclusion

So I made a memorandum about the unexpectedly complicated standard output. Well, it seems unlikely that people who need to make their own progress bar ... Well, I'm sure it's good that you can play various things by using these functions.

reference

-Escape Sequence -Unicode list -Aa of Manuki Kotobuki was completed -2ch cute AA / emoticon summary