The application server uWSGI, which can handle not only Python but also Ruby, php, and Perl, has a mode called cheaper. This is a function that dynamically adjusts the number of workers according to the load.
By default, there were spare, backlog, and busyness control algorithms. However, I didn't like any of them, so I created an algorithm called spare2 and merged it into the original family. Since we've included the master branch (development), we've also provided a stable version of Plugins for uWSGI 2.0.
I will explain the spare2 algorithm added this time and why it was difficult to adopt other existing algorithms. (See Documentation for more information on other algorithms.)
spare2
The explanation is based on the sample configuration file.
[uwsgi]
...
processes=10
cheaper-algo=spare2
cheaper=2
cheaper-initial=2
cheaper-step=2
cheaper-idle=30
process = 10 is the maximum number of worker processes. cheaper = 2 is the minimum number of processes and will be adjusted during this time. cheaper-initial = 2 is the number of workers immediately after startup.
The spare2 algorithm attempts to maintain the number of idle workers specified by cheaper. For example, if the number of idle workers becomes one, 2-1 = 1 worker will be started.
cheaper-step = 2 is the maximum number of processes that can be started at one time. When cheaper is large, you can adjust cheaper-step so that it doesn't start too many processes at once.
If the number of idle processes is greater than cheaper for cheaper-idle = 30 seconds, one worker process is stopped. By setting this value longer, you can prevent workers from stopping and restarting in detail.
spare
[uwsgi]
...
processes=10
cheaper-algo=spare
cheaper=2
cheaper-initial=2
cheaper-step=2
cheaper-overload=5
The spare algorithm counts when there are no processes in the idle state (= overload) and when there are two or more processes in the idle state (= idle). If you count up one, the other will be reset, but if you have only one idle process, neither will be counted.
When the overload count reaches cheaper-overload = 5, it adds cheaper-step = 2 worker processes.
When the idle count reaches cheaper-overload = 5, stop one worker process.
This algorithm requires a small cheaper-overload and a large cheaper-step to quickly increase the number of workers, such as when a server is added under load distribution. Still, the initial response time tends to be significantly worse because new workers are not launched until all worker processes are busy.
Also, if you set cheaper-overload = 1, the number of workers tends to increase or decrease little by little. For example, if you have about 40 workers, it feels overkill to stop a worker with just two idle processes.
backlog
It is similar to spare, but it makes an overload judgment by taking advantage of the fact that Linux can monitor the number of connections accumulated in the TCP backlog.
It's out of the question because it inherits the weaknesses of spare and isn't available when nginx and uWSGI are connected on unix sockets.
busyness
Busyness complicates spare.
Let cheaper-overload be the time frame, and let busyness be the percentage of workers who were idle during that time. Set the minimum and maximum values of this busyness, and if it falls below the minimum value, the number of workers will decrease, and if it exceeds the maximum value, the number of workers will increase.
Other options are quite complicated, such as the option to make it difficult to reduce the number of workers, and the option to monitor the backlog and add workers without waiting for the time frame, but this also increases the number of workers smoothly at the beginning. It is difficult to handle the inrush load.
Recommended Posts