[PYTHON] Reinventing the Wheel: libMerge to merge function definitions in Bash library into ShellScript


In this article, I will describe libMerge, a Python 3.x script I made. libMerge has the ability to merge function definitions in the Bash library into ShellScript. To be precise, only the definition part of the function used in Script is extracted from the Bash library, Create a new Script that describes "extracted function + original Script contents".

The reason for creating libMerge is as follows.

  1. Bash does not have a general-purpose library, and there is no common understanding of the functions of the functions in the library. Therefore, it is preferable to provide a Script that makes it easy for a third party to refer to the function definition.
  2. I don't want to manually post library function definitions to Script for third parties.
  3. Scripts that are not complete in a single file are less portable. If the library cannot be included, or if the library is modified, it will not work.

By the way, in 2005, in the article "shsubrmerge (CodeZine) to share and merge shell scripts", A Script with almost the same function is published. It's a reinvention of the wheel. Furthermore, there seems to be no well-known package with equivalent functionality, so It is expected that there will be no demand. ~~ It's good because I can solve my problem (・ word ・) ~~

table of contents

  1. Install libMerge
  2. Reference library: bashLib
  3. libMerge options
  4. How to use libMerge
  5. libMerge processing (mechanism)  6. Limitation

Install libMerge

Since it is published on Github, you can get it by the following procedure (Debian assumption). The libMerge installation directory is under "/ usr / local / bin /".

$ git clone https://github.com/nao1215/BashScriptsCompilation.git -b master
$ cd BashScriptsCompilation
$ sudo ./installScripts         #Installation script

Sample library: About bashLib

It seems that few people have a Bash library, so I prepared a sample library bashLib. BashLib will also be installed by the installation procedure above.

The functions that exist in bashLib are as follows.

Function name Description
errMsg Error message displayed in red on standard output
warnMsg Display error message in yellow on standard output
checkArgc Check the number of arguments
isFile Check if the file exists
isDir Check if the directory exists
isSubshell script is"source"Check if it is executed by the command
isNumeric Check if the variable is quantified
isNull variable(String)Check if is Null
isRoot Check if the user is root
++ variable(Numerical value)Increment
-- variable(Numerical value)Decrement
AplusB Add the specified value to the variable
AminusB Subtract the specified value to the variable
_sum variable(Numerical value)Add
makeDir Create a directory after various checks
makeEmptyFile After various confirmations, create a 0Byte file
getLineNum Get the number of lines in the file
dos2unix Convert files from windows format to unix format

libMerge options

option Description
-l(--lib) Describe the PATH to the Bash library.
-s(--script) To Script that wants to give function definition in library
Describe the PATH.
-o(--output) Script that posted the function definition(Create New)Name
If you want to change it, enter the file name.

How to use libMerge

As an example, prepare the following sample script (sample.sh). In sample.sh, makeDir () and errMsg () defined in "/ usr / local / bin / bashLib" are used. To use these functions, you need to include the library with the "source" command.


# Description : Sample Script

source /usr/local/bin/bashLib  #Library include
makeDir "dir"                  #Library function
errMsg "Sample"             #Same as above

Execution result.

$ ./sample.sh 
make directory dir

Use libMerge to add a function definition to the above sample.sh. Note that instead of inserting the function definition directly into sample.sh, a new Script called m_sample.sh will be generated. The name of the new Script here can be changed with the "-o" option.

$ ls
$ libMerge -s sample.sh -l /usr/local/bin/bashLib
$ ls
m_sample.sh  sample.sh

The generated m_sample.sh is as follows. The function definition is inserted under the Script header, and only the function definition used in Script (sample.sh) is posted.


# Description : Sample Script
function errMsg() {
    local message="$1"
    echo -n -e "\033[31m\c"  # Escape sequence to make text color red 
    echo "${message}"
    echo -n -e "\033[m\c"   # Escape sequence to restore font color

function makeDir() {
    local dir_path="$1"

    checkArgc "$#"
    if [ -d "${dir_path}" ]; then
        echo "Directory "${dir_path}" already exists. Not make it."
        mkdir -p $1
        if [ "$?" = 0 ]; then
            echo "make directory "${dir_path}""

source /usr/local/bin/bashLib  #Library include

makeDir "dir"
errMsg "Sample"

libMerge processing (mechanism)

The following is an overview of the libMerge process.

  1. Option check (error notification if there is a defect)
  2. Parse the Bash library and create a list of start and end line numbers in the function definition
  3. Get the name of a function that exists in the Bash library
  4. Based on the result of step 2, get the definition part of the function for each function. Manage definitions in a dictionary using function names as key values
  5. Parse the Script and delete the functions other than those used in the Script from the list and dictionary.
  6. Insert dictionary element (function definition) under Script header while creating Script copy (m_ \ <script_name >)

Limitation -There is room for improvement in the extraction of the function definition part (especially the definition end part). -The function to delete the library include processing that exists in the original Script is not implemented yet. ・ We have not conducted sufficient tests.


"Because the goal of 2017 is" package creation (C language) ", let's implement libMerge in C language." I thought about such a thing at the time of design (sleeping on February 17, 2011), but I implemented it in Python and it was the correct answer.

Also, the reason why it is difficult to make Bash into a library is "Lack of return value" "The nature of Bash that makes full use of redirect pipes" I experienced it with the creation of libMerge and bashLib this time. The advantages of creating a library are about "improving readability of one-liner (shell art)" and "integrating template processing". If you try to create a good library that demands more than that, you're likely to get bogged down.

Recommended Posts

Reinventing the Wheel: libMerge to merge function definitions in Bash library into ShellScript
Attempt to extend a function in the library (add copy function to pathlib)
Feel free to turn Python using the library into an AWS Lambda function
How to use the C library in Python
A memorandum to register the library written in Hy in PyPI
How to debug the Python standard library in Visual Studio
In omegaconf, let's pass the direct parameter file to the function
I tried to implement the mail sending function in Python
Notes on how to use marshmallow in the schema library