[Feature-length poem] I don't understand functional languages Even if you understand Python: Part 2 The functions that generate functions are amazing.

Introduction

This is a ** feature-length poem ** that describes the features of functional programming in Python. I wrote it so that you can understand it without knowing Python, so please read it if you are PHP, Java, or JavaScript.

【goal】

[Writing motive]

[Target readers]

[Series article]

Function to generate a function

Review: What is a higher-order function?

Higher-order functions are functions that treat the function as data. In particular:

Generate another function within a function

As I explained last time, functions are also objects in Python, so they can be treated as data. So you can ** generate a function inside a function **. For example:

def create_func():
  def newfunc(i):      #Create a new function inside the function
    if i % 2 == 0:
      return "even"
    else:
      return "odd"
  return newfunc       #Return it

##Try using
fn = create_func()     #Create a new function and assign it to the variable fn
print(fn(2))           #2 is an even number"even"Is output
print(fn(3))           #Because 3 is odd"odd"Is output

Functions that generate and return functions in this way are also called higher-order functions. In addition, the function created in the function is sometimes called "intrafunction function" (hereinafter, it is called "inner function" for easier understanding).

In other books, it is explained as "a function that returns a function" instead of "a function that creates a function". But ** it's more important to create a new function than to return a function **. So, in this poem, I will explain it as "a function that generates a function".

Refer to local variables from inside to outside

The following two functions are both very similar. The only difference is the value returned when it is even and the value returned when it is odd.

def create_func1():
  def newfunc(i):
    if i % 2 == 0:
      return "even"    #When even"even"return it
    else:
      return "odd"     #When odd"odd"return it
  return newfunc

def create_func2():
  def newfunc(i):
    if i % 2 == 0:
      return "#fcc"    #When even"#fcc" (Light red)return it
    else:
      return "#ccf"    #When odd"#ccf" (Light blue)return it
  return newfunc

Let's share these two similar functions.

def create_func(even_val, odd_val):  #Add arguments to outer function
  def newfunc(i):      #Inner function(Generated function)Is ...
    if i % 2 == 0:
      return even_val  #When even_Changed to return val
    else:
      return odd_val   #When odd_Changed to return val
  return newfunc

There are two points:

By doing this, you can ** generate a number of functions with slightly different behaviors ** as follows:

## "even"When"odd"Generate a function that returns
fn1 = create_func("even", "odd")
print(fn1(0))    #=> even
print(fn1(1))    #=> odd
print(fn1(2))    #=> even
print(fn1(3))    #=> odd

## "#fcc"When"#ccf"Generate a function that returns
fn2 = create_func("#fcc", "#ccf")
print(fn2(0))    #=> #fcc
print(fn2(1))    #=> #ccf
print(fn2(2))    #=> #fcc
print(fn2(3))    #=> #ccf

Let's compare this with the functions in the previous section.

The important point is that ** the inner function is accessing the local variables (arguments in this case) of the outer function **. Such a function is called a "closure". This is explained in the next section.

Accessing the local variables of an inner function from an outer function is generally not possible in any language, not just Python.

Summary so far

  • Functions and objects in Python
  • → You can create a new function in a function
  • The inner function can access the local variables (including arguments) of the outer function
  • → You can easily generate a function with slightly different behavior.

closure

What is a closure?

What is a closure? According to Wikipedia:

Closures (closures), function closures are a type of function object in programming languages. In some languages, it is realized by lambda expressions and anonymous functions. It is characterized by resolving variables other than arguments in the environment (static scope) in which it is defined, not in the environment at runtime. It can be said that it is a pair of a function and an environment that evaluates it. This concept can be traced back to at least the 1960s SECD machines.

(... omitted ...)

Closures are a mechanism for sharing an environment within a program. Lexical variables differ from global variables in that they do not occupy the global namespace. It also differs from an object instance variable in that it is bound to a function call rather than an object instance.

Well, I don't know what it is. It seems that you need about 145 IQ to understand this explanation.

For the time being, think of closures as ** inner functions ** accessing local variables of outer functions ** (which may be academically inaccurate, but for practical purposes this level of understanding is sufficient. ).

For example, in the functions in the previous section, the inner function is accessing the local variables of the outer function (Note: the argument is also one of the local variables). Therefore, the inner function is a closure.

##This is the outer function
def create_func(even_val, odd_val):  #Note that the arguments are also local variables
  ##This is the inner function
  ##  (This function is a closure because we are accessing an outer local variable)
  def newfunc(i):
    if i % 2 == 0:
      return even_val     #Accessing an outer local variable
    else:
      return odd_val      #Accessing an outer local variable
  return newfunc

In contrast, the next case is not a closure. This is because the inner function is not accessing the outer local function.

def create_func(even_val, odd_val):
  ##This is not a closure
  ## (Because the inner function is not accessing the outer local variable)
  def newfunc(i):
    if i % 2 == 0:
      return "even"
    else:
      return "odd"
  return newfunc

Also, the following cases are not closures. This is because the inner function is accessing a global variable.

##These are global variables
even_val = "#fcc"
odd_val  = "#ccf"

def create_func():
  ##This is also not a closure
  ## (I'm accessing a global variable)
  def newfunc(i):
    if i % 2 == 0:
      return even_val
    else:
      return odd_val
  return newfunc

How closures work

To get a deeper understanding, let's take a look at how closures work.

From an implementation point of view, ** closures are "functions with variable tables" **. The difference between closures and regular functions is the presence or absence of this variable table.

The following is an example in Python3. You can see that the variable table is stuck to the closure.

##Function that returns a closure
def create_func(even_val, odd_val):
  def newfunc(i):
    if i % 2 == 0: return even_val
    else:          return odd_val
  return newfunc   # (Not just a function)Return closure

##When you create a closure ...
fn = create_func("red", "blue")
## __closure__You can see that the variable table is attached
print(fn.__closure__[0].cell_contents)   #=> red
print(fn.__closure__[1].cell_contents)   #=> blue

##It seems that you can not change it arbitrarily
fn.__closure__[0].cell_contents = "white"
   #=> AttributeError: attribute 'cell_contents' of 'cell' objects is not writable

Also, normal functions (not closures) don't have this variable table.

##A function that returns a normal function
def create_func():
  def newfunc(i):
    if i % 2 == 0: return "even"
    else:          return "odd"
  return newfunc   # (Not a closure)Returns just a function

##For normal functions, there is no variable table
fn = create_func()
print(fn.__closure__)    #=> None

As you can see, the closure has a variable table attached to it. In other words, a ** closure is a state-based function **. It's often said that "functions have no state", but note that closures don't.

Accessing the outer local variables is via the variable table, which is slower than accessing regular local variables. It's just like accessing an instance variable is slower than accessing a local variable.

<Addition 1> He commented, "Is it implementation-dependent that is slower than local variables?" That's true, it's slower than local variables in Python, but it's quite possible that it's not in other languages and processors. Thank you for pointing out. </ Addition 1>

<Addition 2> He pointed out that "Isn't it good to say that closures have" state "? It seems better to say" attribute "or" environment "" (Thank you). Personally, I think the expression "state" is fine for the following reasons. ・ The word "state" does not include whether or not it can be updated. ・ It is just a matter of perspective to make "state" an "attribute" (a function can be regarded as a calculation formula for returning a value and a discriminant for returning a boolean value, but it is still a function anyway. Same as) ・ "Environment" is a term that you can see in technical books, but its definition is vague and confuses beginners, and the substance is nothing more than a variable table. However, what I call a "state" may be something that beginners call Wikipedia a Wiki. If you are interested, here, series, [tweet]( Please see https://twitter.com/tanakahisateru/status/606098308411453442). </ Addition 2>

Imitate closures with objects

By the way, have you ever heard the explanation that "an object is a variable with a function attached to it"? If a variable has a function attached to an object and a function has a variable attached to a closure **, then the object and closure are likely to be very similar.

In fact, ** closures can be imitated by objects **. The following is an example.

##The function that returns closures from the previous section
def create_func(even_val, odd_val):
  def func(i):
    if i % 2 == 0:
      return even_val
    else:
      return odd_val
  return func

##Make a class that imitates it
class CreateFunc(object):

  def __init__(self, even_val, odd_val):  #constructor
    self.even_val = even_val
    self.odd_val  = odd_val

  def __call__(self, i):
    if i % 2 == 0:
      return self.even_val
    else:
      return self.odd_val

##How to use
obj = CreteFunc("red", "blue")   #Create object(Note 1)
print(obj.__call__(0))    #=> red
print(obj.__call__(1))    #=> blue
print(obj.__call__(2))    #=> red
print(obj.__call__(3))    #=> blue
##This is fine(Note 2)
print(obj(0))             #=> red
print(obj(1))             #=> blue
print(obj(2))             #=> red
print(obj(3))             #=> blue

## (Note 1)In Python, you don't need the new operator when creating an object.
##You can create an object just by calling the class as if it were a function.
## (Note 2)Obj in python.__call__()Obj()Can be called like.

In this way, what the closure is doing can be imitated by the object. This means that languages that don't support closures can do the same thing as closures.

However, closures are more concise (in this case), as you can see in the following comparison.

##Closure version##Object version
def create_func(even, odd):       class CreateFunc(object):
                                    def __init__(self, even, odd):
                                      self.even = even
                                      self.odd  = odd

  def func(i):                      def __call__(self, i):
    if i % 2 == 0:                    if i % 2 == 0:
      return even                       return self.even
    else:                             else:
      return odd                        return self.odd

  return func

fn = create_func("red", "blue")   obj = CreateFunc("red", "blue")
fn(0)                             obj.__call__(0)   # or obj(0)
fn(1)                             obj.__call__(1)   # or obj(1)

As you can see from the above, closures and objects are very similar. If there is an IQ145 beautiful girl who insists that "object-oriented programming that retains state is wrong! Functional language is correct!", "But seniors, functional language closures are very similar to objects, aren't they? Including the point of having a state. " I'm sure IQ145 will give you a great answer.

Summary so far

  • Closures are inner functions that access local variables of outer functions.
  • The entity is a function object with a variable table
  • Note that even the inner function is not a closure if you are not accessing the local variables of the outer function
  • Closures and objects are similar
  • Closure: A function with data stuck to it
  • Object: A function attached to data

Wrapper function

"Higher-order functions that generate functions" are very useful when creating wrapper functions. Here, we will explain the "wrapper function" to show the convenience of "higher-order functions that generate functions".

What is a wrapper function?

** A wrapper function is a function that adds another process when calling an existing function **. For example:

  • (A) Add or modify arguments before calling the original function
  • (B) Call the original function and then work on the return value
  • (C) Add pre-processing and post-processing when calling the original function

Of course, there are other possibilities, but for the time being, you should keep the above three types.

Let's look at a concrete example.

table = [
  {"name": "Haruhi", "size": "C"},
  {"name": "Mikuru", "size": "E"},
  {"name": "Yuki",   "size": "A"},
]

##Original function
def search(name):             #Function to retrieve data
  for d in table:
    if d["name"] == name:
      return d
  return None

## (A)Add or tweak arguments before calling the original function
def search1(name):
  if name == "Michiru":
    name = "Mikuru"           #Crafting arguments
  return search(name)         #Call the original function

## (B)Call the original function and then work with the return value
def search2(name, targets=["Mikuru"]):
  ret = search(name)          #Call the original function
  if name in targets:
    ret["size"] = "It is a prohibited matter"   #Craft the return value
  return ret

## (C)Add pre-processing and post-processing when calling the original function
def search3(name):
  print("** name=%s" % name)  #Preprocessing
  ret = search(name)          #Call the original function
  print("** ret=%s" % ret)    #Post-processing
  return ret

In this way, the wrapper function allows you to craft arguments, craft return values, and add processing before and after. In other words, a wrapper function is a function added to the original function.

Higher-order function that generates a wrapper function

Now let's write a higher-order function that will generate these wrapper functions. The points are as follows.

  • ** Point 1: Take the original function as an argument and **
  • ** Point 2: Returns a new function or closure with added functionality **
def normalize(func):           #Receive the function
  def wrapper(name):
    if name == "Michiru":
      name = "Mikuru"          #After crafting the arguments
    return func(name)          #Call the original function
  return wrapper               #Functions like(closure)return it.

def censor(func, targets):     #Receive the function
  def wrapper(name):
    ret = func(name)           #After calling the original function
    if name in targets:
      ret["size"] = "Prohibitions" #Craft the return value
    return ret
  return wrapper               #Functions like(closure)return it.

def trace(func):               #Receive the function
  def wrapper(name):
    print("** name=%s" % name) #Add pre-processing
    ret = func(name)           #Call the original function
    print("** ret=%s" % ret)   #Add post-processing
    return ret
  return wrapper               #Functions like(closure)return it.

The usage of these is as follows.

table = [
  {"name": "Haruhi", "size": "C"},
  {"name": "Mikuru", "size": "E"},
  {"name": "Yuki",   "size": "A"},
]

def search(name):            #Function to retrieve data
  for d in table:
    if d["name"] == name:
      return d
  return None

##Wrapper function to craft arguments
search1 = normalize(search)

##Wrapper function to craft the return value
search2 = censor(search, ["Mikuru"])

##Wrapper function with pre-processing and post-processing added
search3 = trace(search)

Higher-order functions make it very easy to generate a wrapper function with added functionality.

Not only that. You can easily combine these functions freely. For example:

##Wrapper function that crafts both arguments and return values
search12 = censor(normalize(search), ["Mikuru"])

##Wrapper function with specially crafted arguments and return values, with pre- and post-processing added
search123 = trace(censor(normalize(search), ["Mikuru"]))

Or:

##Wrapper function with specially crafted arguments and return values, with pre- and post-processing added
search = normalize(search)
search = censor(search, ["Mikuru"])
search = trace(search)

Why are you able to combine freely? That's because higher-order functions take the original function as an argument.

###For example, in the first example, the original function was embedded.
###Therefore, the function is fixed and cannot be changed to another function.
def search3(name):
  print("** name=%s" % name)
  ret = search(name)    #Function is fixed
  print("** ret=%s" % ret)
  return ret

###On the other hand, the higher-order function does not have the original function embedded in it.
###Since it is received as an argument, it can be easily changed to another function.
def trace(func):        #Receive the original function as an argument
  def wrapper(name):
    print("** name=%s" % name)
    ret = func(name)    #The function is not fixed!
    print("** ret=%s" % ret)
    return ret
  return wrapper

Partial application

By the way, in functional programming, there is the word "partial application". If you think this is "one of the functions that generate a wrapper function", that's fine.

def add(x, y):        #Example: Suppose you have a function that adds two numbers
  return x + y

def add1(x):          #If you fix one argument to 1, this is
  return add(x, 1)    #It becomes a "function that adds 1 to a certain number".

def adder(y):         #If you want to specify an arbitrary number instead of fixing it to 1.
  def wrapper(x):     #Such a higher-order function may be used.
    return add(x, y)
  return wrapper

add3 = adder(3)       #Example of use(add(x, y)Of which y is fixed at 3)
print(add3(7))        #=> 10

def apply(func, y):   #Let's make it possible to specify the function itself.
  def wrapper(x):
    return func(x, y)
  return wrapper

add3 = apply(add, 3)  #Example of use(add(x, y)Of which y is fixed at 3)
print(add3(7))        #=> 10

Creating a new function with some arguments given from a certain function in this way is called "partial application".

However, partial application is not often used in Python. Since it is refreshing to curry, the explanation is omitted.

Summary so far

  • A wrapper function is a function that adds functionality to the original function.
  • (A) Add or modify arguments before calling the original function
  • (B) Call the original function and then work on the return value
  • (C) Add pre-processing and post-processing when calling the original function
  • Higher-order functions make it easy to generate wrapper functions
  • Point 1: Take the original function as an argument
  • Point 2: Returns a new function or closure with added functionality
  • "Partial application" is one of the methods to generate a wrapper function.
  • Example: Based on ʻadd (x, y), generate a function ʻadd3 (x) equivalent to ʻadd (x, 3)`

Function decorator

Python has a feature called "function decorators" to help you use wrapper functions better. In Python, you rarely call "higher-order functions that generate wrapper functions" directly, and you usually use them through this "function decorator". This "function decorator" is an indispensable and important feature for using Python.

What is a function decorator?

By the way, the sample code of the "higher-order function that generates the wrapper function" earlier was like this.

def search(name):
  ....

search = trace(search)

This is correct code, but the function name search has appeared three times. Therefore, when you change the function name, you must change it in at least three places. It is a so-called "not DRY" state.

In such a case, in Python you could write: This is called a ** function decorator **. (It looks like Java annotations, but the contents are completely different.)

@trace               #← This is the function decorator
def search(name):
  ....

With the function decorator, you only have to write the function name in one place (it's very "DRY"). It may seem strange at first, but @trace def search (): ... is the same as def search (): ...; search = trace (search), so Don't think hard.

You can also specify multiple function decorators if desired.

##this is,
@deco1
@deco2
@deco3
def func():
  ....

##Same as this
def func():
  ....
func = deco1(deco2(deco3(func)))

Function decorators are not found in languages other than Python. Therefore, it may look strange, but it should not be difficult if the contents so far.

Function decorator with arguments

Function decorators are difficult when they take arguments. I'll explain it now, but if you don't understand it (if you don't want to study Python in earnest), don't worry and skip it.

First, consider the case where a "higher-order function that receives the original function and returns a new function" takes arguments other than the original function * *.

##This higher-order function has arguments targets other than func
def censor(func, targets):
  def wrapper(name):
    ret = func(name)
    if name in targets:
      ret["size"] = "It is a prohibited matter"
    return ret
  return wrapper

Attempting to use this as a function decorator will result in an error. Do you know why you get an error?

##This will result in an error
@censor
def search(name):
  ...

##Because censor()Because there is no second argument targets of
search = censor(search)
   #=> TypeError: censor() missing 1 required positional argument: 'targets'

The code above is a function decorator, so Python will try to do search = censor (search). However, because the second argument (targets) ofcensor ()is not specified, an error occurs.

To avoid this, create a "function that takes arguments and returns a'function decorator'". A function decorator is a "function that receives a function and returns a new function", which means that you create a "function that takes an argument and returns a'function that receives a function and returns a new function'" (!!).

def censor(targets):    #A function that takes an argument and returns a function decorator
  def deco(func):         #Make a function decorator
    def wrapper(name):      #Create a wrapper function
      ret = func(name)        #Call the original function
      if name in targets:     #Refers to the outermost argument
        ret["size"] = "It is a prohibited matter"
      return ret              #Returns a return value
    return wrapper          #Returns a wrapper function
  return deco             #Returns a function decorator

Wow, it's complicated ...

Here's how to use it.

##this is
@censor(["Mikuru"])
def search(name):
  ....

##Same as this
def search(name):
  ....
deco = censor(["Mikuru"])
search = deco(search)

##Note that this is different!!
search = censor(search, ["Mikuru"])

That's all for the function decorator with arguments, but did you understand?

If you didn't understand: You don't have to worry about it. One day, we will know. If you don't, you'll find that "even if you don't understand the little things, it doesn't have a big impact on your life."

Those who understand: I'm sure you just think you understand. When that kind of "I think I understand" is perfect, I write poems that are genius and beautiful girls. Let's watch out.

Python's function decorators are certainly useful if you can master them. However, how to create it differs depending on whether it takes an argument or not, and defining a function decorator that takes an argument is very cumbersome and difficult to teach. To be clear, ** Python's function decorators are a design mistake **, like this.

Just by allowing the function decorator to accept non-function arguments, these problems can be solved at once. Why don't you do this ...

##For example, if you could write like this
@censor ["Mikuru"], "It is a prohibited matter"
def search(name):
  ....

##If this ↑ becomes the same as this ↓
def search(arg):
  ....
search = censor(search, ["Mikuru"], "It is a prohibited matter")

##It's written in the same way as a function decorator that takes arguments, and a decorator that doesn't.
##Even if you change it to take a function decorator that takes no arguments, the compatibility on the user side is
##I wish everyone could be happy because they were kept.
def censor(func, targets, seal):
  def wrapper(name):
    ret = func(arg)
    if name in targets:
      ret["size"] = seal
    return ret
  return wrapper

</ End>

Do you need a function decorator?

I haven't seen the function decorator feature outside of Python (it may be, but it's not popular). As a result, users in other languages may say, "This is just syntactic sugar for higher-order functions, isn't it necessary?"

However, the function decorator function is a very nice feature that you don't have to **. That's because the function decorator feature makes some code very readable.

For example, consider the following JavaScript code:

var compile = taskdef("*.html", ["*.txt"], function() {
  var html = this.product;
  var txt  = this.materials[0];
  system.run("txt2html " + html + " > " + txt);
});

If JavaScript had a function decorator feature, this could be written: Don't you think this one is much easier to read?

@taskdef("*.html", ["*.txt"])
function compile() {
  var html = this.product;
  var txt  = this.materials[0];
  system.run("txt2html " + html + " > " + txt);
}

Also, consider another code like this: function () {...} is multiple nested.

function test_it_should_exit_with_0_when_same_contents() {
  with_dummyfile("A.txt", "aaa", function() {
    with_dummyfile("B.txt", "aaa", function() {
      status = system.run("compare A.txt B.txt");
      asssert_equal(status, 0);
    });
  });
};

If you could use a function decorator in JavaScript, you could probably write: It's very refreshing because there is no multiple nesting.

@with_dummyfile("B.txt", "aaa")
@with_dummyfile("A.txt", "aaa")
function test_it_should_exit_with_0_when_same_contents() {
  status = system.run("compare A.txt B.txt");
  asssert_equal(status, 0);
};

The function decorator is just syntactic sugar, so it's easy to implement, but the effect you get is pretty big. It is a cost-effective function for languages, so I hope it will be introduced in other languages as well.

Is it different from the decorator pattern?

One of the design patterns in class design is the "decorator pattern". I think the term "function decorator" in Python probably comes from this pattern name. We also often use function decorators * features * as a means for the purpose of decorators * patterns *.

But there is a difference between the two.

  • ** The "decorator pattern" is not a language function, but just one of the patterns **. The "decorator pattern" is designed so that the new function or class can be used in the same way as the original function or class (= ** has the same interface **). This will replace the original function or class with a new one.

  • Python's ** "function decorator" feature is one of the finest language features (albeit just syntactic sugar) **. Also, in the "function decorator", the original function and the new function do not have to be used in the same way (= ** interfaces may be different **). It's common to add or subtract arguments to the original function, or change the data type of the return value.

You can see that Python's function decorators are different from the decorator pattern. If true, it should have been a more accurate name for the feature.

Summary so far

  • "Function decorator function" is syntactic sugar to make higher-order functions that generate wrapper functions easier to use.
  • @deco def fn (): ... is the same as def fn (): ...; fn = deco (fn)
  • @deco (arg) def fn (): ... is the same as def fn (): ...; fn = deco (arg) (fn)
  • @ deco1 @ deco2 def fn (): ... is the same as def fn (): ...; fn = deco1 (deco2 (fn))
  • Different from "decorator pattern"
  • Decorator pattern ... One of the design patterns (not a language feature). The purpose is to add functions up to the same usage (= same interface) as the original function or class.
  • Function decorator: One of the language functions, the substance is just syntactic sugar. The interface may be different from the original function or class, and in fact it is often the case.

Other Python-specific stories

Change the outer local variable

When you access a local variable of an outer function from an inner function in Python, you can read the variable without any preparation, but there are restrictions on writing the variable.

##For Python3
def outer():
  x = 10
  y = 20
  def inner():
    x = 15      #This is inner()Is assigned to a local variable of.
    nonlocal y  #But by using nonlocal,
    y = 25      # outer()Can be assigned to a local variable of.
  inner()
  print(x)   #=> 10  (Not changed)
  print(y)   #=> 25  (has been changed)

outer()

##For Python2
def outer():
  x = 10
  y = [20]      #Python2 doesn't have nonlocal, so you need this trick
  def inner():
    x = 15      #This is inner()Is assigned to a local variable of.
    y[0] = 25   #Note that this hasn't changed the variable y!
  inner()
  print(x)      #=> 10  (Not changed)
  print(y[0])   #=> 25  (y has not changed, but y[0]Has been changed)

outer()

It's a little different compared to other languages. There are many reasons why Python has such a specification.

  • JavaScript requires var to declare local variables. Thanks to that, the specification can be "local variable of inner () if var x = 15, local variable of outer () if x = 15".
  • In Ruby, the grammar of function / method definition and closure definition is different. Thanks to that, x = 15 is" a local variable of inner () if it is in a function / method definition. "" If it is in a closure definition, use the variable x in outer () if it exists, otherwise inner ( ) Is treated as a local variable. "
  • Python doesn't have var, and both functions and closures are done with def, so the syntax is the same. Therefore, when changing the local variable from inner () to outer (), it is necessary to specify it using nonlocal.
In principle, even if the grammar of the function definition and closure definition is the same, it is possible to distinguish between them, so as with Ruby, the specification is that if there is a local variable with the same name on the outside, it will be assigned to it. You should be able to. In principle.

Function decorators and class decorators

In addition to function decorators, Python also has a feature called "class decorators."

@publish              #← This is the class decorator
class Hello(object):
  ...

##This is the same as
class Hello(object):
  ...
Hello = publish(Hello)

In Python, the term "decorator" usually refers to a function decorator. However, there are two types of decorators, and they can be confused with one of the design patterns (as mentioned above), the "decorator pattern", so it's best to call them "function decorators" as much as possible to avoid ambiguity. ..

Arbitrary arguments

Previous function decorator sample code assumed that the original function had only one argument.

def trace(func):
  def newfunc(name):
    print("** name=%s" % name)
    ret = func(name)    #← Assuming only one argument
    print("** ret=%s" % ret)
    return ret
  return newfunc

Therefore, this function decorator can only be applied to functions that have only one argument.

@trace
def search(name):   #← Applicable because there is only one argument
  for d in tables:
    if d["name"] == name: return d
  return None

@trace
def add(x, y):      #← Because there are two arguments, a run-time error occurs.
  return x+y

add(10, 20)
  #=> TypeError: newfunc() takes 1 positional argument but 2 were given

To make this applicable to any number of arguments:

def trace(func):
  def newfunc(*args):    #← Receive any number of arguments
    print("** args=%r" % (name,))
    ret = func(*args)    #← Pass any number of arguments
    print("** ret=%r" % (ret,))
    return ret
  return newfunc

@trace
def add(x, y):      #← No error occurs even with two arguments
  return x+y

add(x, y)

To accommodate arbitrary keyword arguments, do the following:

def trace(func):
  def newfunc(*args, **kwargs):    #← Receive arbitrary keyword arguments
    print("** args=%r, %kwargs=%r" % (name, kwargs))
    ret = func(*args, **kwargs)    #← Pass arbitrary keyword arguments
    print("** ret=%r" % (ret,))
    return ret
  return newfunc

@trace
def add(x, y):
  return x+y

add(y=20, x=10)     #← Calling using keyword arguments is OK

Function name and function documentation

When you define a function in Python, the function object is populated with the function name and documentation.

def search(name):
  """Search table by name."""
  for d in table:
    if d["name"] == name:
      return d
  return None

##View function name and documentation
print(search.__name__)  #=> search
print(search.__doc__)   #=> Search table by name.

But with higher-order functions and function decorators, they disappear or have different names.

def trace(func):
  def newfunc(name):
    print("** name=%s" % name)
    ret = func(name)
    print("** ret=%s" % ret)
    return ret
  return newfunc

@trace
def search(name):
  ....

print(search.__name__)   #=> newfunc  (The function name has changed!)
print(search.__doc__)    #=> None (The document has disappeared!)

This is not very good. Therefore, it is a good idea to copy the function name and documentation from the original function to the new function.

def trace(func):
  def newfunc(name):
    ....
  #Copy the function name and documentation from the original function
  newfunc.__name__ = func.__name__
  newfunc.__doc__  = func.__doc__
  return newfunc

@trace
def search(name):
  ....

print(search.__name__)   #=> search
print(search.__doc__)    #=> Search table by name.

A utility that makes this copy is provided as standard in Python, so it is a good idea to use it.

import functools

def trace(func):
  @functools.wraps(func)   #← Copies from the original function
  def newfunc(name):
    ....
  return newfunc

@trace
def search(name):
  ....

print(search.__name__)   #=> search
print(search.__doc__)    #=> Search table by name.

Summary

In this poem, we explained "functions that generate functions" among higher-order functions. You can use it to generate functions that behave slightly differently. The important thing here is the "closure", which is a function with variables attached to it.

I also explained that a "wrapper function" that adds functionality to an existing function is useful. Higher-order functions are also useful when generating wrapper functions, and we've also introduced a feature called "function decorators" to make them easier to read.

I think that IQ145 can explain it more easily than a beautiful girl senior, but if you notice anything, please comment.

Exercises

I will write an example of the answer in the comment section.

【Question 1】 Consider a function that alternates between two values each time you call it.

## 'red'When'blue'A function that alternately returns
print(toggle_color())  #=> 'red'
print(toggle_color())  #=> 'blue'
print(toggle_color())  #=> 'red'
print(toggle_color())  #=> 'blue'

##A function that alternates between 1 and 0
print(toggle_on_off())  #=> 1
print(toggle_on_off())  #=> 0
print(toggle_on_off())  #=> 1
print(toggle_on_off())  #=> 0

## 'show'When'hide'A function that alternately returns
print(toggle_visible())  #=> 'show'
print(toggle_visible())  #=> 'hide'
print(toggle_visible())  #=> 'show'
print(toggle_visible())  #=> 'hide'

Define a higher-order function new_toggle () that produces such a function. How to use:

toggle_color   = new_toggle("red", "blue")
toggle_on_off  = new_toggle(1, 0)
toggle_visible = new_toggle("show", "hide")

** [Problem 2] ** Let's define a function that takes an array of strings and converts them all to uppercase and lowercase using the higher-order function map () (for map (), previous. kwatch / items / 03fd035a955235681577))::

def uppers(strings):
  return map(lambda s: s.upper(), strings)

def lowers(strings):
  return map(lambda s: s.lower(), strings)

print(uppers(['a', 'b', 'c']))   #=> ['A', 'B', 'C']
print(lowers(['A', 'B', 'C']))   #=> ['a', 'b', 'c']

Also, let's define a function to find the length of all strings using map () in the same way:

def lengths(strings):
  return map(lambda s: len(s), strings)
  ##Or return map(len(s), strings)

print(lengths(["Haruhi", "Mikuru", "Yuki"]))  #=> [6, 6, 4]

All of these have the same code, except for the lambda expression.

So, try defining a higher-order function mapper () that easily generates these functions. How to use:

uppers = mapper(lambda s: s.upper())
lowers = mapper(lambda s: s.lower())
lengths = mapper(lambda s: len(s))   #Or mapper(len)

print(uppers(['a', 'b', 'c']))   #=> ['A', 'B', 'C']
print(lowers(['A', 'B', 'C']))   #=> ['a', 'b', 'c']
print(lengths(["Haruhi", "Mikuru", "Yuki"]))  #=> [6, 6, 4]

** [Problem 3] ** Suppose you have a function that sends an HTTP request, like this:

##Function to send an HTTP request
def http(method, url, params={}, headers={}):
  ....

##Function to send an HTTPS request
def https(method, url, params={}, headers={}):
  ....

##How to use
response = http("GET", "http://www.google.com/", {"q": "python"})

I made a wrapper function for this for each of the GET / POST / PUT / DELETE / HEAD methods.

def GET(url, params={}, headers={}):
  return http("GET", url, params, headers)

def POST(url, params={}, headers={}):
  return http("POST", url, params, headers)

def PUT(url, params={}, headers={}):
  return http("PUT", url, params, headers)

def DELETE(url, params={}, headers={}):
  return http("DELETE", url, params, headers)

def HEAD(url, params={}, headers={}):
  return http("HEAD", url, params, headers)

But this is a similar code iteration. Is there a way to write it more concisely?

** [Problem 4] ** The following code is test code written in Python. You can see that the dummy file is generated / deleted in the test.

import os, unittest

class FileTest(unittest.TestCase):

  def test_read(self):
    ##Create a dummy file
    filename = "file1.txt"
    File.open(filename, "w") as f:
      f.write("FOO\n")
    ##Run the test
    try:
      with open(filename) as f:
        content = f.read()
      self.assertEqual(content, "FOO\n")
    ##Delete dummy file
    finally:
      os.unlink(filename)

To write this more concisely, define a decorator function called @with_dummyfile ():

import os, unittest

class FileTest(unittest.TestCase):

  @with_dummyfile("file1.txt", "FOO\n")   #← Define this
  def test_read(self):
    with open("file1.txt") as f:
      content = f.read()
    self.assertEqual(content, "FOO\n")

** [Problem 5] ** A framework for a web application was used in the following way. I use a function decorator called @view_config (), but it seems to be long to read and write.

##It's too long to read and write
@view_config(request_method="GET",
             route_urlpath="/api/books/{book_id}/comments/{comment_id}")
def show_book(self):
  book_id    = self.request.matched['book_id']
  comment_id = self.request.matched['comment_id']
  ....

So define a function decorator called @ on () that can be written more concisely. How to use:

@on("GET", "/api/books/{book_id}/comments/{comment_id}")
def show_book(self):
  book_id    = self.request.matched['book_id']
  comment_id = self.request.matched['comment_id']
  ....

Also, define a decorator @ api that accepts the parameters in the URL path pattern as function arguments. How to use:

@on("GET", "/api/books/{book_id}/comments/{comment_id}")
@api
def show_book(self, book_id, comment_id):
  #Argument book_id is self.request.matched['book_id']Is passed.
  #Argument commend_id is self.request.matched['comment_id']Is passed.
  ....

If you can, incorporate the functionality of @ api into@ on (). How to use:

@on("GET", "/api/books/{book_id}/comments/{comment_id}")
def show_book(self, book_id, comment_id):
  #Argument book_id is self.request.matched['book_id']Is passed.
  #Argument commend_id is self.request.matched['comment_id']Is passed.
  ....

Tip:

##this is
keyword_args = {"book_id": "123", "comment_id": "98765"}
show_book(self, **keyword_args)
##Same as this
show_book(self, book_id="123", comment_id="98765")

Recommended Posts