[PYTHON] Easily daemonized with Supervisor

Thing you want to do

You may want to run it as a daemon process, such as a script that you want to start all the time. However, for example, if you write the script of /etc/init.d yourself ...

In other words, it should be about the same ** "It is troublesome to write a process management script" ** If you register it as a service, it will be ** "Basically start / stop management as root" **, so You want to avoid these two.

With systemd you don't have to write a script,

I think there is demand even in such cases.

deamonize tool

For this kind of problem, you will have to use a daemonization tool. Daemon tools or forever. Here, I will use "supervisor" implemented in python. Because it is stable.

supervisor

As a test

Installation

There are two main methods.

  1. Install from EPEL repository
  2. Use the python package manager (pip / easy_install).

Method 1 may be out of date. If you want to use the latest version, use method 2.

1. When installing from EPEL

yum install supervisor
rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt
rpm -ivh http://apt.sw.be/redhat/el6/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm

2. Use pip / easy_install.

If you install from a python package manager such as pip / easy_install, Unlike installing from the EPEL repository, by default it does not include service management for supervisord itself. It's a good idea to do that manually.

sudo easy_install supervisor

#Or

sudo pip install supervisor

If you don't have a python package manager in the first place To use easy_install, use sudo yum install python-setuptools, To use pip, run sudo easy_install pip after installing easy_install.

The EPEL repository installation will automatically place it in /etc/supervisord.conf, but you will generate it manually.

sudo echo_supervisord_conf > /etc/supervisord.conf

/Etc/supervisord.d is automatically generated by EPEL repository installation, so create it.

sudo mkdir /etc/supervisord.d

Make adjustments around the log file that the supervisord main unit spits out. The default config is /tmp/supervisord.log, so Modify /etc/supervisord.conf to make it /var/log/supervisor/supervisord.log.

[supervisord]
#Log directory
;logfile=/tmp/supervisord.log
logfile=/var/log/supervisor/supervisord.log

Create a log output destination directory, and then set log rotation.

sudo mkdir /etc/supervisord.d
sudo sh -c "echo '/var/log/supervisor/*.log {
       missingok
       weekly
       notifempty
       nocompress
}' > /etc/logrotate.d/supervisor"

Also modify /etc/supervisord.conf. Make sure the pid file is generated under / var / run / Allows you to include configuration files under /etc/supervisord.d/.

[supervisord]
# ...Omission
#pid file
;pidfile=/tmp/supervisord.pid
pidfile=/var/run/supervisord.pid

# ...Omission

#Since the include section is commented out, comment it in and modify it as follows.
[include]
files = supervisord.d/*.ini

It seems that some people have created the init script, so let's thank you for using it. https://github.com/Supervisor/initscripts

It supports several types of distros, but this time I will use the following.

https://github.com/Supervisor/initscripts/blob/master/redhat-init-equeffelec


sudo curl -o /etc/rc.d/init.d/supervisord https://raw.githubusercontent.com/Supervisor/initscripts/master/redhat-init-equeffelec
sudo chmod 755 /etc/rc.d/init.d/supervisord
sudo chkconfig --add supervisord

In CentOS7, you will be registered with systemd. Create the systemd unit file by referring to the following.

https://github.com/zokeber/supervisor-systemd/blob/master/etc/systemd/system/supervisord.service

Store it in /etc/systemd/system/supervisord.service with the following contents.

[Unit]
Description=Supervisor process control system for UNIX
Documentation=http://supervisord.org
After=network.target

[Service]
ExecStart=/usr/bin/supervisord -n -c /etc/supervisord.conf
ExecStop=/usr/bin/supervisorctl $OPTIONS shutdown
ExecReload=/usr/bin/supervisorctl $OPTIONS reload
KillMode=process
Restart=on-failure
RestartSec=50s

[Install]
WantedBy=multi-user.target

After creating it, check the management registration to systemd with systemctl list-unit-files --type = service and try starting it.

systemctl start supervisord
systemctl status supervisord
systemctl stop supervisord

If there is no problem, register for automatic startup.

systemctl enable supervisord.service

supervisord config

[inet_http_server]
port=127.0.0.1:9001

[supervisorctl]
serverurl=unix:///var/tmp/supervisor.sock
serverurl=http://127.0.0.1:9001

Process management registration

The supervisor I installed is version 3 series, and the config seems to be slightly different from the 2 series. Especially around the log output. (2 series is old, so let's use 3 series)

Below, the standard output log storage directory for jobs is set to / var / log / supervisor / jobs. If you want to use it as it is, create it in advance. Also, the following rotation settings are appropriate.

[program:GrowthForecast]
command=/usr/local/bin/growthforecast.pl --enable-float-number --data-dir /data/gf    ;Start command
user=monadmin  ;Launch user
autorestart=true  ;Automatic restart when process goes down
stdout_logfile=/var/log/supervisor/jobs/GrowthForecast-supervisord.log ;Standard output log
stdout_logfile_maxbytes=1MB
stdout_logfile_backups=5
stdout_capture_maxbytes=1MB
redirect_stderr=true  ;Redirect error output to standard output

Also create this in /etc/supervisord.d/GrowthForecast.ini.

[program:HRForecast]
command=perl hrforecast.pl --config config.pl  ;Relative path from directory is OK.
user=monadmin
directory=/opt/HRForecast  ;If you have an installation directory, go here.
autorestart=true
stdout_logfile=/var/log/supervisor/jobs/HRForecast-supervisord.log
stdout_logfile_maxbytes=1MB
stdout_logfile_backups=5
stdout_capture_maxbytes=1MB
redirect_stderr=true

Also create this in /etc/supervisord.d/HRForecast.ini as well.

Start confirmation

sudo service supervisor start

It's a good idea to check the logs under / var / log / supervisor. With this setting, the stdout / err of the set process (GrowthForecast or HRForecast) is also displayed.

After starting, try hitting the client command supervisorctl from someone other than root. Hit it alone to enter the command line management interface. So, if successful, it looks like this:

[[email protected]]% supervisorctl
GrowthForecast                   RUNNING    pid 24955, uptime 0:12:00
HRForecast                       RUNNING    pid 24956, uptime 0:12:00
supervisor>

From the CLI console, start|stop|restart [Process name]、 A list of management processes will appear in status.

Supervisorctl without entering the CLI console{start|stop|restart|status} [Process name]Etc. are possible.

You can do it without becoming root, You now have a daemon that can be managed from the user level.

If you fail

[[email protected]]% supervisorctl
GrowthForecast                   FATAL      can't find command 'growthforecast.pl'
HRForecast                       RUNNING    pid 8136, uptime 0:00:29
supervisor>

If the startup fails, the status will be FATAL. It was this that I wrote a command that does not go through the PATH.

What I haven't done & other

Recommended Posts

Easily daemonized with Supervisor
Easily beep with python
Easily serverless with Python with chalice
Easily write if-elif with lambda
Easily build CNN with Keras
Easily cProfile with a decorator
Easily implement subcommands with python click
Easily handle lists with python + sqlite3
Easily switch pyenv virtualenvs with peco
Easily handle databases with Python (SQLite3)
Easily build virtual machines with Vagrant
[Python] Collect images easily with icrawler!
Easily post to twitter with Python 3
Easily send emails with Gmail with Django
Easily draw a map with matplotlib.basemap
Easily build a development environment with Laragon
Daemonize a Python web app with Supervisor
Easily try automatic image generation with DCGAN-tensorflow
Easily implement ItemView incremental search with PySide
Easily download mp3 / mp4 with python and youtube-dl!
Easily implement login authentication function with Laravel