I don't want to take the time to put it together, so it's just like a memo.
> lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 7.6 (wheezy)
Release: 7.6
Codename: wheezy
In another case, Supervisor is set to `` `3.0r1-1 ~ bpo70 + 1``` which is sourced from sid. As with Ubuntu 12.04, the original 3.0a8 seems to be quite old (2010). Ubuntu 14.04 is 3.0b2, a little new (2013?)
Is it 3.1.2 now? …… Release date, 2014-09-07 is 3 days before (at the time of writing this article)
Reference: http://supervisord.org/changes.html
At first, I was playing with Django with the following feelings.
I think it's okay, but it's a little aggressive
I wanted to do something about it. I want to reduce the dependence on OS standard packages.
Supervisor is not the only pip here because I want something like /etc/init.d/supervisor. pip does not include OS-side configuration files that follow the distribution style. Thank you to the distribution.
Another test project in the development environment A tornado project (started as a reverse proxy from Apache) There seems to be no problem in managing all at once with the Supervisor attached to this apt, so Do it like this.
So, the point is, you need to "use Supervisor to make Celery workers resident via Django in venv".
The original (dj) celery setting is in `/ etc / default / celeryd```. Unfortunately this is effectively a bash file called from bash ( `/etc/init.d/celeryd```), so
Must be converted to Supervisor settings.
When doing this, I will write down the points that I needed to pay attention to, so I hope it will give you some hints.
There is no point in including CELERY_OPTS etc. in the Supervisor settings.
In other words, for something like a constant in /etc/init.d/celeryd
The two types are mixed together. The only distinction is to look at the source.
Especially if you have multiple (Celery) workers I write the settings of queues and workers in `` `/ etc / default / celeryd```. celeryd_multi doesn't seem to work very well with Supervisord.
djcelery's celeryd_multi Actually, Celery's celery-multi Is executed while inheriting Django's settings (as you can see from celeryd_multi.py).
Celery's celery-multi command is "start worker".
Even with a simple "Django + Celery" configuration There is no such thing as "... / manage.py celery-multi" in the residual process that can be seen in ps etc. It means that celery-multi is not a daemon. Instead, "... / manage.py celery worker" should be alive as multiple daemons.
With that feeling, trying to run manage.py celeryd_multi on Supervisor seems to be useless.
Reference: http://stackoverflow.com/questions/15558875/running-celeyd-multi-with-supervisor
/etc/default/celery
ENABLED="true"
CELERYD_LOG_LEVEL="DEBUG"
CELERY_TIMEZONE='Asia/Tokyo'
CELERYD_NODES="manage build"
CELERYD_CHDIR="/opt/griflet/"
CELERYD_MULTI="$CELERYD_CHDIR/manage.py celeryd_multi"
CELERYCTL="$CELERYD_CHDIR/manage.py celeryctl"
#XXXX should be a number
CELERYD_OPTS="--time-limit=XXXX --time-limit:build=XXXX -c 1 -Q:build build -Q manage"
CELERY_CONFIG_MODULE="myproject.celeryconfig"
CELERYD_LOG_FILE="/var/log/celery/%n.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYD_USER="myproject"
CELERYD_GROUP="myproject"
export DJANGO_SETTINGS_MODULE="myproject.settings"
Then, it looks like the following. However, it has not been verified.
text:conf.d/myproject.conf
[program:celery-myproject-manage]
command=/opt/myproject/venv/bin/python /opt/myprojcet/manage.py celery worker
--loglevel=DEBUG
-Q manage
--logfile=/var/log/celery/manage.log
--pidfile=/var/run/celery/manage.pid
--hostname manage@hostname
--concurrency 1
--time-limit=XXXX
--workdir=/opt/myproject/
user=myprojcet
directory=/home/myprojcet
numprocs=1
stdout_logfile=/var/log/celery/manage.stdout.log
stderr_logfile=/var/log/celery/manage.stderr.log
autostart=true
autorestart=true
startsecs=10
environment =
DJANGO_SETTINGS_MODULE="myprojcet.settings",
CELERY_TIMEZONE="Asia/Tokyo"
[program:celery-myprojcet-build]
command=/opt/myprojcet/venv/bin/python /opt/myprojcet/manage.py celery worker
--loglevel=DEBUG
-Q build
--logfile=/var/log/celery/build.log
--pidfile=/var/run/celery/build.pid
--hostname build@hostname
--concurrency 1
--time-limit=XXXX
--workdir=/opt/myprojcet/
user=myprojcet
directory=/home/myprojcet
numprocs=1
stdout_logfile=/var/log/celery/build.stdout.log
stderr_logfile=/var/log/celery/build.stderr.log
autostart=true
autorestart=true
startsecs=10
environment =
DJANGO_SETTINGS_MODULE="myprojcet.settings",
CELERY_TIMEZONE="Asia/Tokyo",
It seems that SUPERVISOR is included in the worker process environment variable that works in this way. This may be useful if you want to change the behavior in a Celery-only configuration and a Supervisor-only configuration. You can use it for debugging.
After that, if you read the manual, there should be a correct description from this article.
Check that the Supervisor is started by stopping Celery with chkconfig or sysv-rc-conf.
That's it.
I was in trouble because the handling of group changed between the case of using celeryd_multi and the above method, so that is also a word. GID specified by CELERYD_GROUP created by celeryd_multi cannot be specified by Supervisor. The user's GID is simply reused.
The celery worker has a way to specify --uid and --gid, so you can actually change the UID and GID of the process. However, in order for this to work effectively, it is necessary to first stop specifying the user of Superviser (start it as root), and also Because the log file is generated "before" the worker eats that option The permissions of the newly created log will be root: root. When using RotatingFileHandler etc., at one point it suddenly becomes impossible to write from WSGI and it falls. Be careful if you have some ideas for managing log file permissions.
In my case, if the user is broken and GID = nogroup and I change to Supervisor, For some reason, all the files created belong to the no group group and I don't understand the meaning.
This is a Unix-like story rather than a Supervisor story overall, but just in case.