Overview

When multithreading was performed using Thread.new (), a resource shortage error occurred on Heroku.

can't create Thread: Resource temporarily unavailable (ThreadError)

We created a thread error handling and a retry mechanism that uses threads on a best effort basis.

By the way, the number of process threads that can be executed on Heroku is quite limited, so be careful about the difference from the local environment.

https://devcenter.heroku.com/articles/limits#processes-threads

As of 2020/08

free, hobby and standard-1x dynos support no more than 256
standard-2x and private-s dynos support no more than 512
performance-m and private-m dynos support no more than 16384
performance-l and private-l dynos support no more than 32768

If your local environment is MacOS, you can use the command sysctl kern.num_taskthreads to find out the maximum number of threads per process.

$ sysctl kern.num_taskthreads
kern.num_taskthreads: 4096

Caution

There is a high possibility that a design mistake is made in the process of running out of threads when the load is slightly higher than usual.

First consider how to reduce the number of threads that must be set up, such as whether processing can be delegated to another server or whether there is a way to do it with a small number of requests due to API specifications.

code

def retry_threads(times: 3)
  try = 0
  begin
    try += 1
    Thread.new { yield }
  rescue ThreadError
    sleep(1 * try)
    retry if try < times
    raise
  end
end

If no thread is available, wait a few seconds and retry.

The number of seconds to wait is variable, such as 1 second for the first retry and 2 seconds for the second retry, to prevent time loss.

There is also a way to manage retries in detail by setting the waiting time to microseconds.

Actual usage

def heavy_task(url)
  #Heavy processing
end

# urls = ["...","...",...]
threads = []
urls.all each do |url|
  threads << retry_threads{ heavy_task(url) }
end
threads.each(&:join)

benchmark

We investigated how much the tolerance for threading was actually increased by this retry mechanism.

conditions

--Processing that takes 10 seconds is subject to retry --Increase the retry waiting time by 1 second --Up to 3 retries --Run in local environment (Mac OS Catalina) --Maximum number of threads: 4096 --Set the number of tasks that can be processed to the benchmark value

Measurement code

def heavy_task
  sleep(10)
end

def retry_threads(times: 3)
  try = 0
  begin
    try += 1
    Thread.new { yield }
  rescue ThreadError
    sleep(1 * try)
    retry if try < times
    p $count
    raise
  end
end

def no_retry_threads()
  begin
    Thread.new { yield }
  rescue ThreadError
    p $count
    raise
  end
end

$count = 0

#No retry
loop do
  no_retry_threads{ heavy_task }
  $count += 1
end

#With retry
loop do
  retry_threads{ heavy_task }
  $count += 1
end

result

The number of tasks processed that took 10 seconds in the local environment (maximum number of threads: 4096).

No retries	With retry
4094	212888

reference

class Thread -Method to catch exceptions in Ruby and retry a certain number of times
Maximum Number Of Threads (Linux/MacOS) + OutOfMemory unable to create new native thread -What is the difference between a process and a thread? --One program can have one process and one or more threads per process

[Ruby / Rails] Mechanism for retrying Thread Error