[Bash] Use here-documents to get python power from bash

Introduction

Parsing files with shell is pretty tough. You can do your best to write using regular expressions, but most of the time you forget the content. There was a case where I was able to get out of the trouble of regular expressions with the help of python modules. This is the record at that time.

environment

• OS
  • CentOS 7.4
• bash
  • GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
• Python
  • Python 3.6.8

What should I do specifically?

Use bash here-documents to pour python code into python commands.

shellscript

python <<EOF

#python code

EOF

I will not write about the bash here document here. If you want to know more, please check by yourself.

sample

Below is a sample. It is a shell script that uses python's configparser module to parse the conf file for mysql and connect to MySQL.

connect_mysql.sh

#!/usr/bin/env bash
#
# Connect to MySQL

declare -r COLOR_RED="\e[33;41;1m"
declare -r COLOR_OFF="\e[m"
declare -r DB_CONFIG_PATH='path/to/database.conf'
# check file exists or not
if [ ! -f "${DB_CONFIG_PATH}" ]; then
  echo -e "${COLOR_RED}[error] \"${DB_CONFIG_PATH}\" does not exist.${COLOR_OFF}"
  exit 1
fi

declare -r SECTION=$1
# check argument
if [ -z "$SECTION" ]; then
  echo -e "${COLOR_RED}[error] Please set section name as command argument.${COLOR_OFF}"
  exit 1
fi


#######################################
# Parse DB config by python
# Globals:
#   DB_CONFIG_PATH
#   SECTION
# Arguments:
#   None
# Returns:
#   None
#######################################
function parse_ini_file () {

  python3 <<'EOF' - "${DB_CONFIG_PATH}" "${SECTION}"

import sys
import configparser

file_path    = sys.argv[1]
section_name = sys.argv[2]

config = configparser.ConfigParser()

config.read(file_path)

try:
    details = config[section_name]
except KeyError:
    sys.exit(1)

db_option = \
    '[client]\\nhost={host}\\nport={port}\\ndatabase={database}\\nuser={user}\\npassword={password}\\n'\
    .format(\
        host=details['host'],\
        port=details['port'],\
        database=details['database'],\
        user=details['user'],\
        password=details['password']\
    )

sys.stdout.write(db_option)
sys.exit(0)

EOF


}

db_option=$(parse_ini_file)

if [ $? -ne 0 ]; then
  echo -e "${COLOR_RED}[error] Section \"${SECTION}\" does not exist in config file.${COLOR_OFF}"
  exit 1
fi

mysql --defaults-extra-file=<(echo -e ${db_option})

It seems that you can do something like python file name argument 1 argument 2 even in a here document by doing the following.

python3 <<'EOF' - "${DB_CONFIG_PATH}" "${SECTION}"
...
EOF

in conclusion

I hope it helps someone.