[PYTHON] EP 9 Consider Generator Expressions for Large Comprehensions

• LIst comprehensions can cause problems for large inputs by using too much memory.

• Generator expressions avoid memory issues by producing outputs one at a time as an iterator.
• Generator expressions can be composed by passing the iterator from one generator expression into the for subexpression of another.
• Generator expressions execute very quickly when chained together.

Effective python

Generator is lazy evaluation and comprehension is eager evaluation.

>>> comprehension = [x ** x for x in range(10)]
>>> comprehension
[1, 1, 4, 27, 256, 3125, 46656, 823543, 16777216, 387420489]

>>> generator = (x ** x for x in range(10))
>>> generator
<generator object <genexpr> at 0x10afeea98>
>>> next(generator)
1
>>> next(generator)
1
>>> next(generator)
4
>>> next(generator)
27
>>> next(generator)
256
>>> next(generator)
3125
>>> next(generator)
46656
>>> next(generator)
823543
>>> next(generator)
16777216
>>> next(generator)
387420489
>>> next(generator)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

yakshaving Different type of evaluation strategy

• Eager evaluation or greedy evaluation

• eager evaluation, an expression is evaluated as soon as it is bound to a variable.

• Lazy evaluation

• delays the evaluation of an expression until its value is needed (non-strict evaluation) and which also avoids repeated evaluations (sharing).

• Partial evaluation

• Remote evaluation

• a general term for any technology that involves the transmission of executable software code from a client computer to a server computer for subsequent execution at the server.

• e.g. RPC

• Short-circuit evaluation

• the semantics of some Boolean operators in some programming languages in which the second argument is executed or evaluated only if the first argument does not suffice to determine the value of the expression Benefit of lazy evaluation

• Performance increases by avoiding needless calculations, and error conditions in evaluating compound expressions

• The ability to construct potentially infinite data structures

• The ability to define control flow (structures) as abstractions instead of primitives

Generator can make a chian of iterator but be careful when you execute it more than once

>>> it = (x * x for x in range(10))
>>> roots = ((x, x ** 0.5) for x in it)
>>> next(it)
0
>>> next(it)
1
>>> next(it)
4
>>> next(it)
9
>>> next(roots)
(16, 4.0)
>>> next(roots)
(25, 5.0)
>>> next(roots)
(36, 6.0)
>>> next(roots)
(49, 7.0)
>>> next(it)
64
>>> next(roots)
(81, 9.0)

Copy iter: itertools.tee