[PYTHON] A note about subprocess

This article is old. Updated in another article.

Here. https://qiita.com/HidKamiya/items/e192a55371a2961ca8a4 We have summarized it more broadly, focusing on subprocess.run () and subprocess.Popen (), which are not covered here.

Deprecated commands

--os.system (cmd) #Command execution in subshell (similar to sh -c "cmd" on the command line) #Exit status --. Spawn {lvpe} (mode, path / file) #path execution. mode: os.P _ .. # l / v: Fixed / variable length argument, p: Search in environment variable PATH, e: Take over current process environment -.Popen (cmd, mode) #Communicate to external processes (scripts, software, etc.). Read / write (mode:'r','w') #pipe open? --.Exec {lvpe} (path / file) #process substitution --commands.getstatusoutput (cmd) # (exit status, result) --. Getoutput (cmd) #Result

Overview

--subprocess.call (cmd) #Exit status --.Check_call (cmd) #Exit status but Exception on error (CalledProcessError) --.Check_output (cmd) #Result but Exception on error (CalledProcessError) -.Popen (cmd) # Specify file at input / output

cmd contents: When taking an argument, make it a list or tuple

--cmd.strip (). split ("") Familiar processing. If there is space inevitably, divide it. --shlex.split (cmd) If there is a grouping such as "" in the character string, do not divide it by the space in it.

"ls -l" --> ['ls', '-l'] "echo'Hello world!'"-> ['echo','Hello world!'] # Shlex.split (cmd) only

option

--stdin: Standard input --stdout: standard output --stderr: Standard error output --The following special values --Existing file name, descriptor (positive integer) --None: No redirect --shell: via shell --sh -c Same as "cmd" --Not recommended due to the risk of shell injection. --Execution of complex scripts such as those used for shell scripts. --Enter a single character string instead of a list (in a list, the first element is a command, and the rest are treated as arguments). -If you want special characters such as * and ~ to work (such as "ls * .txt") --Pipe is also possible --timeout: Time-out seconds

Special value

--subprocess.DEVNULL: Specify the standard input / output destination as os.devnull (bit bucket, black hole) --subprocess.PIPE: Pipe specification to standard I / O destination --subprocess.STDOUT: Specify that standard error output is output to the same handle as standard output (only for 2> 1 & .stderr)

Error output integration


subprocess.check_output([cmd],stderr=subprocess.STDOUT)

subprocess.Popen.stdin/stdout/stderr --The return value is the same as open () --You can use .write () or .close () for input. --.Read () and .close () can be used for output.

Getting output through a pipe


subprocess.Popen(cmd, stdout=subprocess.PIPE,shell=True).stdout.readlines()

subprocess.Popen.communicate() --You can also read the output with subprocess.Popen.stdout.read () and subprocess.Popen.stderr.read (). --But not recommended due to output size restrictions. --If you pass it to .communicate (), you will receive (stdout_data, stderr_data).

communicate()Acquisition using


subprocess.Popen([cmd], stdout=subprocess.PIPE).communicate()[0]

--In addition, when giving standard input, specify the argument of communicate (input). --When passing as a text file, open the file with open () and give it in the form read by read ().

communicate()Input using


subprocess.Popen(['cat'], stdin=subprocess.PIPE, stdout=subprocess.PIPE).communicate('Hello world!\n')[0]

communicate()Input 2 using


subprocess.Popen(['cat'], stdin=subprocess.PIPE, stdout=subprocess.PIPE).communicate(open('input.txt','r').read())[0]

Description that does not use shell in pipe


p1 = subprocess.Popen([cmd1], stdout=subprocess.PIPE) #Pipe to output destination
p2 = subprocess.Popen([cmd2], stdin=p1.stdout, stdout=subprocess.PIPE) #Receive p1 for input
p1.stdout.close()
output = p2.communicate()[0] #stdout_get data

Other options and default values

--bufsize = -1: Input / output (line) buffer setting. --executable = None: Specify the replacement program to execute. --preexec_fn = None: File call just before the execution of the child process. #POSIX --close_fds = True: Closes file descriptors other than 0, 1, 2 before executing the child process. #POSIX --cwd = None: Working directory --env = None: Define environment variables in the new process --universal_newlines = False: use locale.getpreferredencoding (False) instead of locale.getpreferredencoding () --startupinfo = None: STARTUPINFO #Windows passed to the underlying CreateProcess function --creationflags = 0: CREATE_NEW_CONSOLE or CREATE_NEW_PROCESS_GROUP #Windows --restore_signals = True: Store all signals set in SIG_IGN in SIG_DFL before child process exec --start_new_session = False: Create setsid () system call in child process before executing subprocess #POSIX --pass_fds = (): File descriptor to keep open between parent and child --encoding = None: Encoding name of standard I / O file --errors = None: Error handling during encoding

Recommended Posts

A note about subprocess
A note about mprotect (2)
A note about KornShell (ksh)
A note about TensorFlow Introduction
A note about [python] __debug__
A note about get_scorer in sklearn
A note about mock (Python mock library)
Note about awk
Just a note
A note about doing the Pyramid tutorial
A memorandum about matplotlib
Note about pointers (Go)
A memorandum about Nan.
A note about the python version of python virtualenv
Data analysis in Python: A note about line_profiler
A note about the new style base class
A note about checking modifiers with Max Plus
Send a signal to subprocess
A memorandum about Python mock
A little more about FIFO
A small note following printf
A note about hitting the Facebook API with the Python SDK
Note
A refreshing story about Python's Slice
A note when gcloud is broken
A sloppy story about Python's Slice
I have a question about whitespace
Note
A small sample note of list_head
A note about connecting Spark to OpenStack Swift-based IBM Object Storage
A note that prints numpy.array nicely
A story about using Python's reduce
Deeper about subprocess (3 series, updated version)
[Translation] A note about structured concurrency .. or rather go statements seem harmful
A note about the functions of the Linux standard library that handles time
A note where a Python beginner got stuck
[Note] Read a file from another directory
A story about remodeling Lubuntu into a Chromebook
I touched "Orator" so I made a note
A Java programmer studied Python. (About type)
A memorandum of understanding about django's QueryDict
A note on enabling PostgreSQL with Django
A story about Python pop and append
Memo about Sphinx Part 1 (Creating a project)
A story about a 503 error on Heroku open
About February 02, 2020 * This is a Python article.
[Note] A story about not being able to break through a proxy with pip