[PYTHON] Treat external commands like functions with sh

shI tried a library that can be used by importing the external command as if it were a function.

>>> from sh import echo
>>> echo('hello', 'world')
hello world

Installation

pipCan be installed using.

$ pip install sh

How to use

To use it, just import the command you want to use from the `` `sh``` module and execute it.

>>> from sh import date
>>> date()
Sunday, February 1, 2015 22:50:13 JST

You can also write:

>>> import sh
>>> sh.date()
Sunday, February 1, 2015 22:50:13 JST

argument

Command arguments can be passed as function arguments.

>>> from sh import echo
>>> echo('hello', 'world')
hello world

option

Options can be passed as keyword arguments.

>>> from sh import curl
>>> curl('http://google.com/', o='out.html', silent=True, L=True)

Options can be passed as function arguments as well as arguments. This should be used for commands where the option must precede the argument.

>>> curl('-o', 'out.html', '--silent', '-L', 'http://google.com/')

Standard input

If you specify a character string in the keyword argument _in, you can pass it as standard input to the command.

>>> from sh import cat
>>> cat(_in='hello')
hello

_inIt is also possible to specify iterable for.

>>> cat(_in=['hello\n', 'world\n'])
hello
world

You can also pass a file object.

>>> cat(_in=open('test.txt'))
hoge
fuga
piyo

redirect

Standard output and standard error can be redirected to a file by specifying the file name in the keyword arguments _out and `` `_err```.

>>> from sh import echo, cat
>>> echo('hello', 'world', _out='hello.txt')

>>> cat('hello.txt')
hello world

_out 、_errIt is also possible to pass a callback function to.

>>> echo('hello', 'world', _out=lambda s: sys.stdout.write(s.upper()))
HELLO WORLD

Output iterator

If you specify the keyword argument _iter, you can get an iterator that returns the result line by line. tail -fIn the case of a command with infinite output such as, the process will not end._iterYou need to get the output from the iterator using.

>>> from sh import seq
>>> seq(1, 3)
1
2
3

>>> for n in seq(1, 3, _iter=True):
...     print n.strip()
... 
1
2
3

pipe

Nesting a function allows you to pipe the output of a function to the input of another function.

>>> from sh import echo, wc
>>> wc(echo('hello'), c=True)
       6

If you nest it too much, it will be difficult to understand, so the `of toolz introduced in Previous article It seems to be good to use it in combination with the `` pipe, thread_first``` function, etc.

>>> pipe('hello', echo, lambda x: wc(x, c=True))
       6

>>> thread_first('hello', echo, (wc, '-c'))
       6

shBy default, the pipe works by passing the result to the next command after the previous command finishes. tail -fIf you execute a command that does not end like, it will not come back, so a keyword argument_pipeI need to tell you what is going on in the pipeline.

>>> from sh import yes, tr
>>> it = tr(yes(), 'y','Y', _iter=True) #NG because the yes command does not end
>>> it = tr(yes(_piped=True), 'y','Y', _iter=True) #This is OK
>>> it.next()
u'Y\n'

Glob

Passing an asterisk as a function argument does not expand it.

>>> from sh import ls
>>> ls('./*') #Because it will not be deployed./*Attempts to find are usually unsuccessful.

Use sh.glob to extract. ** Do not use the standard library glob.glob. ** **

>>> from sh import ls, glob
>>> ls(glob('./*')) # sh.Expand using glob

Background execution

By default, the `sh``` function waits for the process to terminate. If you want to run it in the background, add _bg = True``` to the option. You can wait for the process running in the background to end with the `` wait``` method.

>>> from sh import sleep
>>> sleep(10)              #Block for 10 seconds
>>> p = sleep(10, _bg=True) #Will come back soon
>>> p.wait()               #Wait for the process to end

Exit code and error handling

The exit code of the command can be obtained with exit_code. If it ends normally, it usually becomes 0.

>>> from sh import ls
>>> output = ls('/tmp')
>>> output.exit_code
0

An exception is thrown if the command terminates abnormally. A specific exit code can be captured with ErrorReturnCode_ <exit code>` ``. ErrorReturnCode``` captures all exit codes.

>>> from sh import ls, ErrorReturnCode_1
>>> try:
...     output = ls('/hogehoge')
... except ErrorReturnCode_1:
...     print "return code is 1"
... 
return code is 1

>>> try:
...     output = ls('/hogehoge')
... except ErrorReturnCode as e:
...     print 'cmd:', e.full_cmd
...     print 'returncode:', e.exit_code
...     print 'stdout:', e.stdout
...     print 'stderr:', e.stderr
... 
cmd: /bin/ls /hogehoge
returncode: 1
stdout: 
stderr: ls: /hogehoge: No such file or directory

For commands that return a non-zero exit code even if the command succeeds, specify a list of exit codes when the command succeeds in the keyword argument `` `_ok_code```.

signal

You can signal the process with the following three methods:

If the process ends with a signal, `SignalException_ <signal number>` will occur when `` `waitis executed. Also,exit_code``` is the negative signal number.

>>> from sh import sleep
>>> p = sleep(10, _bg=True)
>>> p.kill()
>>> try:
...     p.wait()
... except ErrorReturnCode as e:
...     print 'cmd:', e.full_cmd
...     print 'returncode:', e.exit_code
...     print 'stdout:', e.stdout
...     print 'stderr:', e.stderr
... 
cmd: /bin/sleep 10
returncode: -9
stdout: 
stderr:

How it works

Since the external commands that can be used differ depending on the environment, it can be seen that `` `sh``` dynamically creates the object corresponding to the external command.

The mechanism for importing arbitrary commands is roughly implemented in the following code. The point is that when importing a module, `sys.modules [__name__]` is replaced with a class that inherits `` `ModuleType```.

mysh.py


import sys
import subprocess
from types import ModuleType

class MySh(ModuleType):
    def __init__(self, self_module):
        self.__name__ = self_module.__name__
                                       
    def __getattr__(self, name):
        def command(*args):
            return subprocess.check_output([name] + list(args))
        return command

mysh = sys.modules[__name__]
sys.modules[__name__] = MySh(mysh)

You can use this module to import and run echo as follows:

>>> from mysh import echo
>>> echo('hello', 'world')
'hello world\n'

Reference material

Recommended Posts

Treat external commands like functions with sh
Scripting with Paver-External Commands
Treat the Interface class like that with Python type annotations