Code that splits an iterator into chunks of iterators in python. I found some code on the net to split the list into chunks, but I couldn't find the code to split the iterator into chunks so I wrote it.
If you want to split a list into chunks from the beginning, you can use an index, and it's easy to split iterators while listing them. However, in order to divide the iterator as iterator, the sub-iterator had to notify the base iterator of StopIteration, which made this method difficult. After all, I'm using a slightly unpleasant method.
What's unpleasant is that I'm using the list exhausted = [False] to pass values out of the subiter. If you normally have it as a bool value, you can not write from inside to outside, so we are using a list that is a variable object. The class is used in a situation similar to the reference implementation of itertools.groupby in the official python documentation. So it may be a python-like implementation.
The feature is that it becomes too huge when listed, and it can be easily used even in cases that affect performance.
def chunk(it, n):
it = iter(it)
exhausted = [False]
def subiter(it, n, exhausted):
for i in range(n):
try:
yield next(it)
except StopIteration:
exhausted[0] = True
while not exhausted[0]:
yield subiter(it, n, exhausted)
Recommended Posts