[Bash] Use here-documents to get python power from bash

Introduction

Parsing files with shell is pretty tough. You can do your best to write using regular expressions, but most of the time you forget the content. There was a case where I was able to get out of the trouble of regular expressions with the help of python modules. This is the record at that time.

environment

What should I do specifically?

Use bash here-documents to pour python code into python commands.

shellscript


python <<EOF

#python code

EOF

I will not write about the bash here document here. If you want to know more, please check by yourself.

sample

Below is a sample. It is a shell script that uses python's configparser module to parse the conf file for mysql and connect to MySQL.

connect_mysql.sh



#!/usr/bin/env bash
#
# Connect to MySQL

declare -r COLOR_RED="\e[33;41;1m"
declare -r COLOR_OFF="\e[m"
declare -r DB_CONFIG_PATH='path/to/database.conf'
# check file exists or not
if [ ! -f "${DB_CONFIG_PATH}" ]; then
  echo -e "${COLOR_RED}[error] \"${DB_CONFIG_PATH}\" does not exist.${COLOR_OFF}"
  exit 1
fi

declare -r SECTION=$1
# check argument
if [ -z "$SECTION" ]; then
  echo -e "${COLOR_RED}[error] Please set section name as command argument.${COLOR_OFF}"
  exit 1
fi


#######################################
# Parse DB config by python
# Globals:
#   DB_CONFIG_PATH
#   SECTION
# Arguments:
#   None
# Returns:
#   None
#######################################
function parse_ini_file () {

  python3 <<'EOF' - "${DB_CONFIG_PATH}" "${SECTION}"

import sys
import configparser

file_path    = sys.argv[1]
section_name = sys.argv[2]

config = configparser.ConfigParser()

config.read(file_path)

try:
    details = config[section_name]
except KeyError:
    sys.exit(1)

db_option = \
    '[client]\\nhost={host}\\nport={port}\\ndatabase={database}\\nuser={user}\\npassword={password}\\n'\
    .format(\
        host=details['host'],\
        port=details['port'],\
        database=details['database'],\
        user=details['user'],\
        password=details['password']\
    )

sys.stdout.write(db_option)
sys.exit(0)

EOF


}

db_option=$(parse_ini_file)

if [ $? -ne 0 ]; then
  echo -e "${COLOR_RED}[error] Section \"${SECTION}\" does not exist in config file.${COLOR_OFF}"
  exit 1
fi

mysql --defaults-extra-file=<(echo -e ${db_option})

It seems that you can do something like python file name argument 1 argument 2 even in a here document by doing the following.


python3 <<'EOF' - "${DB_CONFIG_PATH}" "${SECTION}"
...
EOF

in conclusion

I hope it helps someone.

Recommended Posts

[Bash] Use here-documents to get python power from bash
I want to use jar from python
Use thingsspeak from python
Use fluentd from python
Changes from Python 3.0 to Python 3.5
Changes from Python 2 to Python 3.0
Study from Python Hour7: How to use classes
Use MySQL from Python
Use MySQL from Python
I want to use ceres solver from python
Use BigQuery from python.
Use Tor to connect from urllib2 [Python] [Mac]
Python: Use zipfile to unzip from standard input
Use os.getenv to get environment variables in Python
Use mecab-ipadic-neologd from python
How to use python interactive mode with git bash
I wanted to use the Python library from MATLAB
Post from Python to Slack
python3: How to use bottle (2)
Cheating from PHP to Python
[Python] How to use list 1
Use MySQL from Anaconda (python)
Anaconda updated from 4.2.0 to 4.3.0 (python3.5 updated to python3.6)
How to use Python argparse
Python: How to use pydub
[Python] How to use checkio
Switch from python2.7 to python3.6 (centos7)
Connect to sqlite from python
[Python] How to use input ()
How to use Python lambda
[Python] How to use virtualenv
python3: How to use bottle (3)
python3: How to use bottle
Use e-Stat API from Python
How to use Python bytes
Hit REST in Python to get data from New Relic
Let's use Watson from Python! --How to use Developer Cloud Python SDK
Call Matlab from Python to optimize
Use httpie from windows git bash
Python: How to use async with
Use Stanford Core NLP from Python
How to use SWIG from waf
Link to get started with python
Get data from Quandl in Python
[Python] How to use Pandas Series
Create folders from '01' to '12' with python
Post from python to facebook timeline
How to use Requests (Python Library)
How to use SQLite in Python
[Lambda] [Python] Post to Twitter from Lambda!
Read and use Python files from Python
How to get the Python version
Forcibly use Google Translate from python
How to get started with Python
Connect to utf8mb4 database from python
[Introduction to Python] Let's use pandas
Python (from first time to execution)
Use kabu StationĀ® API from Python
Post images from Python to Tumblr
[Python] How to use list 3 Added
How to use Mysql in python