Effective Python Note Item 15 Know how closures relate to function scope

This is a memo of O'Reilly Japan's book effective python. https://www.oreilly.co.jp/books/9784873117560/ P３1~３５

By knowing the scope of closures, you will be able to write beautiful code.

** Sort example ** Sort the numbers in order, but suppose you have a number you want to prioritize

def sort_priority(values, group):
    def helper(x):
        if x in group:
            return (0, x)
        return (1, x)
    values.sort(key=helper)

numbers = [8, 3, 1, 2, 5, 4, 7, 6]
group = {2, 3, 5, 7}
sort_priority(numbers, group)
print(numbers)

>>>
[2, 3, 5, 7, 1, 4, 6, 8]

If you don't know the concept of closures, it's hard to imagine how they work. There are three points here

1. The closure mechanism allows the helper function to access the group that is the argument of the sort_priority function.
2. Since the python function is a first-class citizen, the helper function can be assigned as the key argument of the sort method.
3. The tuple, which is the return value of the helper function, separates the first subscript into 0 and 1, which separates the priority order.

** Incorporate to determine if it is included in the priority list ** Based on the above code, suppose you want to implement an implementation that returns True if it is included in the priority list.

def sort_priority2(numbers, group):
    found = False
    def helper(x):
        if x in group:
            found = True #It should be True here. .. ..
            return(0, x)
        return(1, x)
    numbers.sort(key=helper)
    return found

found = sort_priority2(numbers, group)
print('Found', found)
print(numbers)

>>>

Found False
[2, 3, 5, 7, 1, 4, 6, 8]

Originally, Found should be True in the calculation, but for some reason it returns False. The reason for this is the closure's ** scope **.

If you don't understand the scope, you will suffer from mysterious behavior

The above code does not return False because the scope of Found stays outside the helper function. In other words

def sort_priority2(numbers, group):
    found = False        #Because found exists in the scope here. ..
    def helper(x):
        if x in group:
            found = True #Don't go to the scope here
            return(0, x)
        return(1, x)
    numbers.sort(key=helper)
    return found

As a result, False that is one scope higher is returned. To avoid this, python3 provides a ** nonlocal ** function. nonlocal pushes the scope out of the closure

def sort_priority3(numbers, group):
    found = False
    def helper(x):
        nonlocal found #Here the scope of found is out of the helper function
        if x in group:
            found =True
            return(0, x)
        return(1, x)
    numbers.sort(key=helper)
    return found

found = sort_priority3(numbers, group)
print('Found', found)
print(numbers)

>>>

Found True
[2, 3, 5, 7, 1, 4, 6, 8]

This is the intended movement.

However, if you use nonlocal in a large-scale function, it will affect the range of unintended parts, so be careful. If you want to be on the safe side, it's better to wrap it in a similar class instead of the nonlocal function.

class Sorter(object):
    def __init__(self, group):
        self.group = group
        self.found = False
        
    def __call__(self, x):
        if x in self.group:
            self.found = True
            return(0, x)
        return(1, x)

sorter = Sorter(group)
numbers.sort(key=sorter)
assert sorter.found is True

>>>
(Exception by assert does not occur)

This one can be used without worrying about the scope.

By the way, nonlocal is not supported in python2.

Conclusion

1. Knowing the scope of closures allows you to write concise code
2. Unintended behavior occurs if you do not understand the scope range
3. python3 can be extended outside the closure by using nonlocal (but it's better to limit it to simple functions only)