Precautions when creating a Python generator

At the end of the article below, I wrote that the generator should be containerized.

-Generate Fibonacci numbers with Python closures, iterators and generators

This time I will explain this.

First, create a generator that generates the Fibonacci number as before. In the comment of the previous article, I was taught a sophisticated writing method, so I will use that.

Python


#generator
def fibonacci_generator():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

#Containerized generator
class FibonacciGenerator(object):
    def __iter__(self):
        a, b = 0, 1
        while True:
            yield a
            a, b = b, a + b

These output the same result.

Python


for fib in fibonacci_generator():
    if fib > 1000:
        break
    print fib,

print

for fib in FibonacciGenerator():
    if fib > 1000:
        break
    print fib,

result


 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987

Note that the above two for statements differ only in the generator. It's tedious to write this code every time the generator changes, so let's make it a function.

Python


def print_fib(fib_iterator, limit = 1000):
    for fib in fib_iterator:
        if fib > limit:
            break
        print fib,
    print

print_fib(fibonacci_generator())
print_fib(FibonacciGenerator())

result


 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987

It was neat to create a function that receives a generator.

Now let's modify this function to display the contents of the generator twice.

Python


def print_fib_twice(fib_iterator, limit = 100):
    print "[1]",
    for fib in fib_iterator:
        if fib > limit:
            break
        print fib,
    print " [2]",
    for fib in fib_iterator:
        if fib > limit:
            break
        print fib,
    print

print_fib_twice(fibonacci_generator())
print_fib_twice(FibonacciGenerator())

result


[1] 0 1 1 2 3 5 8 13 21 34 55 89  [2]
[1] 0 1 1 2 3 5 8 13 21 34 55 89  [2] 0 1 1 2 3 5 8 13 21 34 55 89

How, the generator that is not containerized does not display the contents of the second time. The reason for this is that if you use the generator as it is, the same one will be reused the first time and the second time. The containerized generator will create a new generator for each loop. This is easy to understand if you try the following.

Python


def print_fib_twice2(fib_iterator, limit1, limit2):
    print "[1]",
    for fib in fib_iterator:
        if fib > limit1:
            break
        print fib,
    print " [2]",
    for fib in fib_iterator:
        if fib > limit2:
            break
        print fib,
    print

print_fib_twice2(fibonacci_generator(), limit1 = 100, limit2 = 400)
print_fib_twice2(FibonacciGenerator(),  limit1 = 100, limit2 = 400)

result


[1] 0 1 1 2 3 5 8 13 21 34 55 89  [2] 233 377
[1] 0 1 1 2 3 5 8 13 21 34 55 89  [2] 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377

In this way, if a generator is passed to a function that is used multiple times internally, it will cause an unexpected error. Therefore, it is generally safer to containerize it.

On the contrary, when writing a function that receives an iterator, it is better to check whether it is containerized. You can determine if an iterator is containerized by applying ʻiter ()` twice and checking if different iterators are returned.

Python


def print_fib_twice3(fib_iterator, limit = 100):
    if iter(fib_iterator) is iter(fib_iterator):
        raise TypeError("Must supply a container")

    print "[1]",
    for fib in fib_iterator:
        if fib > limit:
            break
        print fib,
    print " [2]",
    for fib in fib_iterator:
        if fib > limit:
            break
        print fib,
    print

Python


print_fib_twice3(fibonacci_generator())

result


TypeError: Must supply a container

Python


print_fib_twice3(FibonacciGenerator())

result


[1] 0 1 1 2 3 5 8 13 21 34 55 89  [2] 0 1 1 2 3 5 8 13 21 34 55 89

This will prevent unintended behavior.

Enjoy!

Recommended Posts

Precautions when creating a Python generator
Precautions when pickling a function in python
[Grasshopper] When creating a data tree on Python script
Problems when creating a csv-json conversion tool with python
A memo when creating a directed graph using Graphviz in Python
[Python] Creating a scraping tool Memo
Precautions that must be understood when building a PYTHON environment
When creating a matrix in a list
Precautions when using phantomjs from python
Precautions when using six with Python 2.5
[Python] Chapter 03-01 turtle graphics (creating a turtle)
When writing a program in Python
Precautions when creating a two-dimensional array with all the same values
Creating a simple PowerPoint file with Python
[Python] Use and and or when creating variables
Note when creating an environment with python
Precautions when solving DP problems with Python
Creating a python virtual environment on Windows
python: Creating a ramen timer (pyttsx3, time)
Things to watch out for when creating a Python environment on a Mac
[Python] Generator function
[Python memo] Be careful when creating a two-dimensional array (list of lists)
Error when installing a module with Python pip
Ubuntu18.04.05 Creating a python virtual environment in LTS
CRLF becomes LF when reading a Python file
Python variadic memorandum when inheriting a defined class
Procedure for creating a LineBot made with Python
Precautions when dealing with control structures in Python 2.6
[Python] Precautions when assigning values to multidimensional arrays
Python Note: When assigning a value to a string
Commands for creating a python3 environment with virtualenv
Procedure for creating a Python quarantine environment (venv environment)
Trap trapped when running a Python Windows executable
A memo for creating a python environment by a beginner
[Web development with Python] Precautions when saving cookies
A story when a Python user passes a JSON file
Precautions when creating a view that narrows down records by date and time
A * algorithm (Python edition)
I want to iterate a Python generator many times
[Python] Take a screenshot
Creating a Home screen
Precautions when handling Luigi
Try creating a compressed file using Python and zlib
4. Creating a structured program
A python lambda expression ...
Notes on creating a python development environment on macOS Catalina
A memorandum when writing experimental code ~ Logging in python
Precautions when installing a hierarchical include directory with waf
Precautions when dealing with ROS MultiArray types in Python
Tips and precautions when porting MATLAB programs to Python
Daemonize a Python process
Creating a scraping tool
Things to note when initializing a list in Python
What's in that variable (when running a Python script)
Implementing a generator using Python> link> yield and next ()> yield
Use communicate () when receiving output in a Python subprocess
Zundokokiyoshi (generator) in Python
Precautions when installing fbprophet
Create a Python environment
Creating a dataset loader
Current directory when creating a new one with Jupyter