I tried using toolz which is an extension of the standard library itertools and `` `functools```.
You can install it with pip.
$ pip install toolz
There is also a Cython version that runs faster.
$ pip install cytoolz
toolzThe functions provided by are roughly divided into the following three. this houseitertoolz 、functoolzAre eachitertools 、functoolsProvides functionality equivalent to the extension of.
toolz also provides standard functions such as map, reduce, and filter. If you import these and it is map, itertools.It will be replaced with a function that can handle Iterable like imap. The usage of these functions is almost the same as the original one, so I will omit it.
Below, we will introduce functions that are likely to be used frequently.
Itertoolz
#### **`It provides the equivalent functionality of itertools.[itertools recipe](http://docs.python.jp/2.7/library/itertools.html#itertools-recipes)There are also functions like those listed in.`**
get
getIs a function that retrieves elements from a sequence or dictionary.
You can get the element from the sequence by specifying the index.
>>> from toolz import get
>>> get(1, range(5))
1
You can also pass the key to get the value from the dictionary.
>>> get('a', {'a': 'A', 'b': 'B', 'c': 'C'})
'A'
It is possible to specify a default value when out-of-index is specified or the key does not exist.
>>> get(10, range(5), 0)
0
>>> get('d', {'a': 'A', 'b': 'B', 'c': 'C'}, 'None')
'None'
You can get multiple values by passing an index or key in a list.
>>> get([1, 3, 5], range(5), 0)
(1, 3, 0)
>>> get(['b', 'd', 'a'], {'a': 'A', 'b': 'B', 'c': 'C'}, 'None')
('B', 'None', 'A')
pluck
pluckIsgetTomapReturns the result equivalent to.
>>> from toolz import pluck
>>> mat = [[(i, j) for i in range(5)] for j in range(5)]
>>> for r in mat:
... print r
...
[(0, 0), (1, 0), (2, 0), (3, 0), (4, 0)]
[(0, 1), (1, 1), (2, 1), (3, 1), (4, 1)]
[(0, 2), (1, 2), (2, 2), (3, 2), (4, 2)]
[(0, 3), (1, 3), (2, 3), (3, 3), (4, 3)]
[(0, 4), (1, 4), (2, 4), (3, 4), (4, 4)]
>>> for r in pluck([2, 4], mat):
... print r
...
((2, 0), (4, 0))
((2, 1), (4, 1))
((2, 2), (4, 2))
((2, 3), (4, 3))
((2, 4), (4, 4))
accumulateIsreduceSimilar to, but returns an iterator that returns cumulative.
Since Python 3.2, it is implemented in itertools.
>>> from toolz import accumulate
>>> from operator import add
>>> list(accumulate(add, range(5)))
[0, 1, 3, 6, 10]
>>> xs = [randint(1, 10) for n in range(10)]
>>> xs
[7, 3, 4, 2, 9, 4, 1, 10, 8, 1]
>>> list(accumulate(max, xs))
[7, 7, 7, 7, 9, 9, 9, 10, 10, 10]
groupbyGroups the elements of the sequence by the value of the key function.
countbyCounts the number of elements in each group.
reducebyIn each groupreduceReturns the result equivalent to executing.
>>> from toolz import groupby, countby, reduceby
>>> from operator import add
>>> xs = range(10)
>>> is_even = lambda n: n % 2 == 0
>>> groupby(is_even, xs)
{False: [1, 3, 5, 7, 9], True: [0, 2, 4, 6, 8]}
>>> countby(is_even, xs)
{False: 5, True: 5}
>>> reduceby(is_even, add, xs)
{False: 25, True: 20}
itertools.groupby groups consecutive elements, whereas toolz.groupby groups regardless of the order of elements.
```pycon
>>> import toolz as tz
>>> import itertools as it
>>> xs = range(10)
>>> is_even = lambda n: n % 2 == 0
>>> tz.groupby(is_even, xs)
{False: [1, 3, 5, 7, 9], True: [0, 2, 4, 6, 8]}
>>> [(k, list(g)) for k, g in it.groupby(xs, is_even)]
[(True, [0]), (False, [1]), (True, [2]), (False, [3]), (True, [4]), (False, [5]), (True, [6]), (False, [7]), (True, [8]), (False, [9])]
consAdds an element to the beginning of the sequence.
concatConcatenates the sequences.
concatvIsconcatvIs a variable length argument.
>>> from toolz import cons, concat, concatv
>>> xs = range(10)
>>> cons(-1, xs)
>>> list(concat([xs, xs]))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(concatv(xs, xs))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
partition 、partition_allSplits the sequence into tuples of the specified length. The operation when there is a remainder is different.
partition. The surplus is not output. partition. Output the remainder. Padding with the specified value where there is no value. Partition_all may shorten the last element.partitionbySplits the sequence with the specified function.
>>> from toolz import partition, partition_all, partition by
>>> xs = range(10)
>>> list(partition(3, xs))
[(0, 1, 2), (3, 4, 5), (6, 7, 8)]
>>> list(partition(3, xs, None))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]
>>> list(partition_all(3, xs))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9,)]
>>> list(partitionby(lambda x: x < 5, xs))
[(0, 1, 2, 3, 4), (5, 6, 7, 8, 9)]
sliding_windowOutputs tuples of the specified length, shifting the indexes one by one.
>>> from toolz import sliding_sindow
>>> list(sliding_window(3, range(10)))
[(0, 1, 2), (1, 2, 3), (2, 3, 4), (3, 4, 5), (4, 5, 6), (5, 6, 7), (6, 7, 8), (7, 8, 9)]
merge_sortedTakes multiple sorted sequences as arguments, merges them, and outputs them.
>>> from toolz import merge_sorted
>>> from random import randint
>>> xs = sorted([randint(1, 10) for _ in range(5)])
>>> ys = sorted([randint(1, 10) for _ in range(5)])
>>> zs = sorted([randint(1, 10) for _ in range(5)])
>>> xs
[3, 6, 6, 6, 9]
>>> ys
[3, 4, 5, 7, 8]
>>> zs
[1, 2, 4, 5, 8]
>>> list(merge_sorted(xs, ys, zs))
[1, 2, 3, 3, 4, 4, 5, 5, 6, 6, 6, 7, 8, 8, 9]
Note that if the sequences entered are not sorted, the results will not be in order.
>>> from toolz import merge_sorted
>>> from random import randint
>>> xs = [randint(1, 10) for _ in range(5)]
>>> ys = [randint(1, 10) for _ in range(5)]
>>> zs = [randint(1, 10) for _ in range(5)]
>>> xs
[4, 3, 10, 7, 3]
>>> ys
[7, 8, 1, 10, 2]
>>> zs
[2, 6, 2, 4, 10]
>>> list(merge_sorted(xs, ys, zs))
[2, 4, 3, 6, 2, 4, 7, 8, 1, 10, 7, 3, 10, 2, 10]
If the inputs are not sorted, you can concatenate and sort to get the results in order.
>>> from toolz import concatv
>>> sorted(concatv(xs, ys, zs))
[1, 2, 2, 2, 3, 3, 4, 4, 6, 7, 7, 8, 10, 10, 10]
Combine the two sequences by the value of the key function.
>>> from toolz import join
>>> from toolz.curried import get #Import curried get
>>> carts = [('Taro', 'Apple'), ('Taro', 'Banana'), ('Jiro', 'Apple'), ('Jiro', 'Orange'), ('Sabu', 'Banana'), ('Sabu', 'Banana')]
>>> prices = [('Apple', 100), ('Banana', 80), ('Orange', 150)]
>>> for x in join(get(1), carts, get(0), prices):
... print x
...
(('Taro', 'Apple'), ('Apple', 100))
(('Jiro', 'Apple'), ('Apple', 100))
(('Taro', 'Banana'), ('Banana', 80))
(('Sabu', 'Banana'), ('Banana', 80))
(('Sabu', 'Banana'), ('Banana', 80))
(('Jiro', 'Orange'), ('Orange', 150))
Functoolz
curry
curryYou can use functions to curry.
>>> from tools import curry
>>> from operator import add
>>> curried_add = curry(add)
>>> curried_add(3)(4)
7
curryCan also be used as a decorator.
>>> from tools import curry
>>> @curry
... def add(a, b):
... return a+b
...
>>> add(3)(4)
7
For the functions provided by toolz, toolz.You can get the curried version by importing from curried.
Let's take the case of the `` `map``` function as an example.
```toolz```From```map```If you install and pass only the function, you will get an error that there are not enough arguments.
```pycon
>>> from toolz import map as not_curried_map
>>> list(not_curried_map(lambda x: x + 1)([1, 2, 3]))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: imap() must have at least two arguments.
curryIf you curry using, you will be able to do the following:
>>> list(curry(not_curried_map)(lambda x: x + 1)([1, 2, 3]))
[2, 3, 4]
toolz.If you import from curried, it is curried and can be executed as it is.
```pycon
>>> from toolz.curried import map as curried_map
>>> list(curried_map(lambda x: x + 1)([1, 2, 3]))
[2, 3, 4]
compose
composeYou can use to synthesize multiple functions.
compose(f, g, h)(x)Isf(g(h(x)))Is the same as.
>>> from toolz import compose, curry
>>> from operators import add, mul
>>> compose(curry(mul)(2), curry(add)(1))(3)
8
pipe, thread_first, thread_last
pipeAlsocomposeYou can apply multiple functions to your data in the same way as
composeAnd the order of the arguments is reversed.
Like a shell pipe, it is evaluated from left to right, just like the data flow.
pipe(x, f, g, h)Ish(g(f(x)))It will be.
>>> from toolz import pipe
>>> from toolz.curried import get
>>> pipe('hello world', str.split, get(0), str.upper)
'HELLO'
thread_first 、thread_lastIs given a one-argument functionpipeIt works in the same way as.
>>> from toolz import thread_first, thread_last
>>> from toolz.curried import get
>>> thread_first('hello world', str.split, get(0), str.upper)
'HELLO'
>>> thread_last('hello world', str.split, get(0), str.upper)
'HELLO'
Functions that take two or more arguments can be passed using tuples, and the behavior is different.
thread_firstIn the case of, the result passed from the previous function becomes the first argument,
thread_lastIn the case of, it is the last argument.
>>> thread_first('hello world', str.split, get(0), str.upper, (add, 'WORLD'))
'HELLOWORLD'
>>> thread_last('hello world', str.split, get(0), str.upper, (add, 'WORLD'))
'WORLDHELLO'
memoizeYou can use to memoize.
memoizeCan also be used as a decorator.
>>> def tarai(x, y, z):
... if x <= y:
... return y
... return tarai(tarai(x-1, y, z), tarai(y-1, z, x), tarai(z-1, x, y))
...
>>> tarai(12, 6, 0)
12
>>> t = memoize(tarai)
>>> t(12, 6, 0)
12
>>> from toolz import juxt
>>> from operator import add, mul
>>> juxt(add, mul)(3, 4)
(7, 12)
identityReturns the argument as is.
>>> from toolz import identity
>>> identity(3)
3
doExecutes a function and returns an argument.
Since the execution result of the function is discarded, it is used for purposes such as outputting a log as a side effect.
The following example adds an argument to log.
>>> from toolz import compose
>>> from toolz.curried import do
>>> log = []
>>> map(compose(str, do(log.append)), range(5))
['0', '1', '2', '3', '4']
>>> log
[0, 1, 2, 3, 4]
Dicttoolz
get_inYou can easily refer to a nested dictionary by passing a list of keys as an argument. You can also specify a default value.
>>> from toolz import get_in
>>> d = {"a": {"b": {"c": 1}}}
>>> d
{'a': {'b': {'c': 1}}}
>>> get_in(["a", "b", "c"], d)
1
>>> get_in(["a", "b", "e"], d, 'None')
'None'
update_inYou can easily update a nested dictionary by passing a list of keys as an argument.
Update is done by passing the update function. No changes are made to the original dictionary and the function is applied to return the updated dictionary.
>>> from toolz import update_in
>>> d = {"a": {"b": {"c": 1}}}
>>> update_in(d, ["a", "b", "c"], lambda x: x+1)
{'a': {'b': {'c': 2}}}
>>> d
{'a': {'b': {'c': 1}}}
If the key does not exist, it will be created with the default value.
>>> update_in(d, ["a", "b", "e"], lambda x: x+1, 0)
{'a': {'b': {'c': 1, 'e': 1}}}
Many features are implemented using itertools and `` `functools```, so it is heavy to use as a reference for using those modules.
Recommended Posts