Investigation of task priority reversal phenomenon in Linux

Background

There was a delay in the periodic timer of the process running on embedded Linux, and I was investigating the status of the context switch of the task by acquiring the CPU trace. Then, there was a place where the normal thread exceeded the priority of the real-time thread. The task priority reversal phenomenon is generally thought to be caused by exclusive control of resources, but this time the cause was investigated because the priority reversal phenomenon occurred between threads that do not use the same resource. saw.

Survey results

The task priority reversal phenomenon this time was due to the "bandwidth throttling mechanism" . In Linux, when a real-time thread settles in an infinite loop, it occupies the CPU, and other processing may not be executed. As a workaround, Linux kernel 2.6.25 and later added the ability to protect the CPU from real-time threads that could monopolize the CPU. This function is called the "bandwidth throttling mechanism".

This mechanism can be tuned with the following/proc filesystem parameters.

/proc/sys/kernel/sched_rt_period_us

Define the time considered to be 100% of the CPU bandwidth in microseconds. The default value is 1000000 microseconds.

/proc/sys/kernel/sched_rt_runtime_us

Defines the time devoted to real-time thread execution in microseconds. The default value is 950000 microseconds. Setting this value to -1 causes the bandwidth control mechanism to fail.

Set the value of sched_rt_runtime_us as follows.

$ sudo sysctl -w kernel.sched_rt_runtime_us=980000
kernel.sched_rt_runtime_us = 980000

References

This article was written with reference to the following information.

· Performance Tuning Guide Red Hat 4.2. CPU Scheduling https://access.redhat.com/documentation/ja-jp/red_hat_enterprise_linux/6/html/performance_tuning_guide/s-cpu-scheduler

・ Tech Knowledge RT Throttling and RT Group Scheduling https://blog.longest-road.com/contents/?p=1236

Recommended Posts

Investigation of task priority reversal phenomenon in Linux
[Linux] List of Linux commands used in practice
[Bash / linux] Notes in case of trouble
Put Python3 in Docker container of Amazon Linux2
Summary of Linux (UNIX) commands that appeared in Progate
The story of building the fastest Linux environment in the world