If you try to run a file with a jar embedded in a shell script like embulk or digdag using Python's subprocess with shell = False
>>> import subprocess
>>> subprocess.check_call(["digdag"])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 181, in check_call
retcode = call(*popenargs, **kwargs)
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 168, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 390, in __init__
errread, errwrite)
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1024, in _execute_child
raise child_exception
OSError: [Errno 8] Exec format error
It fails like. This can be avoided by setting shell = True, but I investigated how to pass arguments to the program because it was different when shell was False (default) and when it was True.
If you give a list when shell = False, it will be processed as a command and its arguments. For example
sample.sh
#!/bin/sh
echo "$1", "$2", "$3" > out.txt
Code that calls a shell script called subprocess.check_call
caller_l0.py
import subprocess
cmd = ["./sample.sh", "1starg", "2ndarg", "3rdarg"]
subprocess.check_call(cmd, shell=False)
When you execute
$ python caller_l0.py
$ cat out.txt
1starg, 2ndarg, 3rdarg
You can see that the arguments are passed as intended. Just set it to shell = True and
caller_l1.py
import subprocess
cmd = ["./sample.sh", "1starg", "2ndarg", "3rdarg"]
subprocess.check_call(cmd, shell=True)
When you execute
$ python caller_l1.py
$ cat out.txt
, ,
Where did you go after the second element of the list?
The official documentation (https://docs.python.org/3/library/subprocess.html) describes the args parameter passed to the subprocess as follows:
args is required for all calls and should be a string, or a sequence of program arguments.
Providing a sequence of arguments is generally preferred, as it allows the module to take
care of any required escaping and quoting of arguments (e.g. to permit spaces in file names).
If passing a single string, either shell must be True (see below) or else the string must
simply name the program to be executed without specifying any arguments.
There is no explanation here for the case where list is given with shell = True. If you look a little more
If args is a sequence, the first item specifies the command string, and any additional
items will be treated as additional arguments to the shell itself.
That is to say, Popen does the equivalent of:
Popen(['/bin/sh', '-c', args[0], args[1], ...])
On MacOS, / bin / sh is bash, so if you do man
-c string If the -c option is present, then commands are read from
string. If there are arguments after the string, they are
assigned to the positional parameters, starting with $0.
It says, but I didn't understand it, so I searched for it https://stackoverflow.com/questions/1711970/cant-seem-to-use-bash-c-option-with-arguments-after-the-c I arrived at -option-string and understood that \ $ 0, \ $ 1, ... in the string given to -c would be replaced.
caller_l1_.py
import subprocess
cmd = ["./sample.sh $0 $1 $2", "1starg", "2ndarg", "3rdarg"]
subprocess.check_call(cmd, shell=True)
$ python caller_l1_.py
$ cat out.txt
1starg, 2ndarg, 3rdarg
When I checked the behavior when multiple arguments were passed after sh -c, other than bash, it was the same behavior on dash on Ubuntu and sh on FreeBSD. From that, I wondered if there was any use for it.
$ bash -c '$0 $1, $2, $3' echo '1-0 1-1' 2 3
1-0 1-1, 2, 3
Then, apply the quoted one, and do the same via subprocess.
$ python -c 'import subprocess;subprocess.check_call(["$0 $1, $2, $3", "echo", "1-0 1-1", "2", "3"], shell=True)'
1-0 1-1, 2, 3
However, if this is the case, you can expand the character string from the beginning, while on the other hand
caller_l1__.py
import subprocess
cmd = ["$0 $1 $2 $3", "./sample.sh", "1-1 1-2", "2", "3"]
subprocess.check_call(cmd, shell=True)
Since the one after being replaced is only passed to sample.sh
$ python caller_l1__.py
$ cat out.txt
1-1, 1-2, 2
After all, I wasn't sure where to give multiple arguments after -c.
The behavior when passing a list with shell = True in the Python subprocess is similar to the behavior when passing multiple arguments after sh -c. However, I didn't know how to use it (someone tell me).
When shell = True, I felt that it was straightforward to specify a character string instead of a list.
caller_s1.py
import subprocess
cmd = "./sample.sh 1starg 2ndarg 3rdarg"
subprocess.check_call(cmd, shell=True)
$ python2.7 caller_s1.py
$ cat out.txt
1starg, 2ndarg, 3rdarg
Recommended Posts