Apparently the environment variables are different when running sidekiq as a service on the production server

Overview

When I started sidekiq as a service on EC2 of AWS with systemd, the phenomenon that the environment variable PATH for the Python command that I should have passed did not work occurred, so I investigated it.

Survey

As a result of inserting whoami and commands to check environment variables in various places in the code I noticed that the setting to start the sidekiq service is suspicious because the user running on the sidekiq job and the controller receiving the POST request are different.

By the way, in Rspec and the local environment, the job that failed in sidekiq was also all green, probably because the environment variable was effective for the execution user.

By the way, what is Systemd? </ strong> This article was helpful when I went around.

Getting started with Systemd https://qiita.com/bluesDD/items/eaf14408d635ffd55a18

approach

I decided to rewrite the settings in the sidekiq.service file of systemd.

systemd sidekiq.service file

Added ExecStartPre to execute a shell script that reads environment variables.

/etc/systemd/system/sidekiq.service


[Unit]
Description=sidekiq
After=syslog.target network.target

[Service]
WorkingDirectory=/var/www/hoge-app
ExecStartPre=/bin/sh /var/www/hoge-app/aws/service/sidekiq_exec_start.sh
ExecStart=/root/.rbenv/bin/rbenv exec bundle exec sidekiq -e production

・ ・ ・(Omission)・ ・ ・

[Install]
WantedBy=multi-user.target

Shell script to run with ExecStartPre

After reading .bash_profile in source or setting environment variables in some way such as export, make it possible to use the same environment variables in systemd with set-environment of systemctl.

/var/www/hoge-app/aws/service/sidekiq_exec_start.sh


source /home/ec2-user/.bash_profile

#Apply environment variables to systemd
/bin/systemctl set-environment PYENV_ROOT="$PYENV_ROOT"
/bin/systemctl set-environment PATH="$PATH"

#Python related configuration process
pyenv global 3.8.3
eval "$(pyenv init -)"

#Write the environment variables set at this point to an arbitrary log file.(For debugging)
LOGFILE=/var/www/hoge-app/log/production.log
echo "Sidekiq Service Exec Start Pre --------" >> $LOGFILE
echo "$PYENV_ROOT" >> $LOGFILE
echo "$PATH" >> $LOGFILE
python -V >> $LOGFILE
whoami >> $LOGFILE
echo "-------------------------------------" >> $LOGFILE

Service restart

systemctl daemon-reload 
systemctl restart sidekiq.service

If the service status is active and the Process ExecStartPre description matches the setting, the setting is successful. After that, actually run the application and check it.

[root@ip-*-*-*-* hoge-app]# systemctl status sidekiq.service
● sidekiq.service - sidekiq
   Loaded: loaded (/etc/systemd/system/sidekiq.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2020-10-31 18:28:39 JST; 3s ago
  Process: 7130 ExecStartPre=/bin/sh /var/www/hoge-app/aws/service/sidekiq_exec_start.sh (code=exited, status=0/SUCCESS)
 Main PID: 7731 (bundle)
   CGroup: /system.slice/sidekiq.service
           └─7731 sidekiq 6.1.1 hoge-app [0 of 1 busy]

Recommended Posts

Apparently the environment variables are different when running sidekiq as a service on the production server
When there are environment variables in Java tests
Run autossh as a systemd service on CentOS