[LINUX] Real-Time group scheduling

Originally, it is a part of the Linux Kernel source code, so it will be treated as GPLv2 (recognition that it should be).

https://www.kernel.org/doc/html/latest/index.html

Licensing documentation

The following describes the license of the Linux kernel source code (GPLv2), how to properly mark the license of individual files in the source tree, as well as links to the full license text.

https://www.kernel.org/doc/html/latest/process/license-rules.html#kernel-licensing

https://www.kernel.org/doc/html/latest/scheduler/sched-rt-group.html


Docs » Linux Scheduler » Real-Time group scheduling

  1. WARNING

Fiddling with these settings can result in an unstable system, the knobs are root only and assumes root knows what he is doing.

Tweaking these settings can lead to system instability. Only root has a handle, and root needs to know what he is doing.

Most notable:

  • very small values in sched_rt_period_us can result in an unstable system when the period is smaller than either the available hrtimer resolution, or the time it takes to handle the budget refresh itself.

.. If the interval is shorter than the valid hrtimer resolution, or shorter than the time it takes to update, a very small value for sched_rt_period_us will result in system instability.

  • very small values in sched_rt_runtime_us can result in an unstable system when the runtime is so small the system has difficulty making forward progress (NOTE: the migration thread and kstopmachine both are real-time processes).

If the run time is difficult for the system to make the next progress, a very small value for sched_rt_runtime_us will result in system instability (Note: both the migration thread and kstopmachine are real-time processes).

  1. Overview

1.1 The problem

Realtime scheduling is all about determinism, a group has to be able to rely on the amount of bandwidth (eg. CPU time) being constant.

Realtime scheduling is all about determinism. The group needs to depend on a certain bandwidth (CPU time, etc.).

In order to schedule multiple groups of realtime tasks, each group must be assigned a fixed portion of the CPU time available.

In order to schedule multiple groups of realtime tasks, each group must be assigned a fixed portion of valid CPU time.

Without a minimum guarantee a realtime group can obviously fall short.

Without a minimum guarantee for the realtime group, it can obviously run short.

A fuzzy upper limit is of no use since it cannot be relied upon. Which leaves us with just the single fixed portion.

Fuzzy caps are unreliable and useless. This leaves only one fixed part.

1.2 The solution

CPU time is divided by means of specifying how much time can be spent running in a given period.

CPU time is divided by specifying how much time is allocated for execution.

We allocate this “run time” for each realtime group which the other realtime groups will not be permitted to use.

This "run time" is assigned to each real time group when it is not assigned to any other real time group.

.

Any time not allocated to a realtime group will be used to run normal priority tasks (SCHED_OTHER).

During the time not allocated to realtime gourp, it is used to execute the normal priority task (SCHED_OTHER).

Any allocated run time not used will also be picked up by SCHED_OTHER.

Unused time of the allocated run time is also scraped by SCHED_OTHER.

.

Let’s consider an example: Consider an example.

a frame fixed realtime renderer must deliver 25 frames a second, which yields a period of 0.04s per frame.

The frame fixed realtime render must provide 25 frames per second. This results in a period of 0.04 seconds per frame.

Now say it will also have to play some music and respond to input, leaving it with around 80% CPU time dedicated for the graphics.

Now, if you have to play music or react to input, you have to leave 80% to process the graphics.

We can then give this group a run time of 0.8 * 0.04s = 0.032s.

This group can be given a run time of `0.8 * 0.04s = 0.032s".

.

This way the graphics group will have a 0.04s period with a 0.032s run time limit.

Thus, the graphics group will have a 0.04s cycle and a 0.032s run time limit.

Now if the audio thread needs to refill the DMA buffer every 0.005s, but needs only about 3% CPU time to do so, it can do with a 0.03 * 0.005s = 0.00015s.

Now if the audio thread has to be inserted into the DMA buffer every 0.005 seconds, it only needs 3% of the CPU time, so run at 0.03 * 0.005s = 0.00015s.

So this group can be scheduled with a period of 0.005s and a run time of 0.00015s.

Therefore, this group can be scheduled with a period of 0.005s and a run time of 0.00015s.

.

The remaining CPU time will be used for user input and other tasks.

The remaining CPU time is used for user input and other tasks.

Because realtime tasks have explicitly allocated the CPU time they need to perform their tasks, buffer underruns in the graphics or audio can be eliminated.

You can prevent graphics or audio buffer underruns because the realtime task explicitly allocates the CPU time required to execute each task.

.

NOTE: the above example is not fully implemented yet. We still lack an EDF scheduler to make non-uniform periods usable. Note: The above example is not yet fully implemented. The EDF scheduler for making the daily uniform cycle a usable tail has not yet been implemented.

  1. The Interface

2.1 System wide settings

The system wide settings are configured under the /proc virtual file system:

System-wide settings consist of the / proc virtual file system.

/proc/sys/kernel/sched_rt_period_us:

The scheduling period that is equivalent to 100% CPU bandwidth

The scheduling period equivalent to 100% CPU bandwidth is:

/proc/sys/kernel/sched_rt_runtime_us:

A global limit on how much time realtime scheduling may use.

The overall limit on how much time is allocated for real time scheduling will be exploited.

Even without CONFIG_RT_GROUP_SCHED enabled, this will limit time reserved to realtime processes.

This limits the amount of time reserved for the realtime process, even if CONFIG_RT_GROUP_SCHED is not enabled.

With CONFIG_RT_GROUP_SCHED it signifies the total bandwidth available to all realtime groups.

CONFIG_RT_GROUP_SCHED is used to show the total bandwidth that is valid for all realtime groups.

  • Time is specified in us because the interface is s32. This gives an operating range from 1us to about 35 minutes.

2.2 Default behaviour

The default values for sched_rt_period_us (1000000 or 1s) and sched_rt_runtime_us (950000 or 0.95s).

By default, sched_rt_period_us takes a value of (1000000 or 1s) and sched_rt_runtime_us takes a value of (950000 or 0.95s).

This gives 0.05s to be used by SCHED_OTHER (non-RT tasks). This will give SCHED_OTHER (non-RT task) 0.05s.

These defaults were chosen so that a run-away realtime tasks will not lock up the machine but leave a little time to recover it.

This default is chosen so that the runaway realtime task does not lock the machine and is only a short time away to recover.

By setting runtime to -1 you’d get the old behaviour back. Setting runtime to -1 will reproduce past behavior.

.

By default all bandwidth is assigned to the root group and new groups get the period from /proc/sys/kernel/sched_rt_period_us and a run time of 0.

By default, the root group is allocated all bandwidth and the new group gets the period from / proc / sys / kernel / sched_rt_period_us and runtime 0.

If you want to assign bandwidth to another group, reduce the root group’s bandwidth and assign some or all of the difference to another group.

If you want to allocate bandwidth to other groups, reduce the bandwidth of the root group and allocate some or all to different other groups. .

Realtime group scheduling means you have to assign a portion of total CPU bandwidth to the group before it will accept realtime tasks.

Realtime group scheduling means that a group must be allocated a portion of the total CPU bandwidth before accepting a realtime task.

Therefore you will not be able to run realtime tasks as any user other than root until you have done that, even if the user has the rights to run processes with realtime priority!

Therefore, if the user has permission to run the realtime priority process, he cannot run the realtime task as a non-root user.

2.3 Basis for grouping tasks

Enabling CONFIG_RT_GROUP_SCHED lets you explicitly allocate real CPU bandwidth to task groups.

You can explicitly allocate real CPU bandwidth to the task group by enabling CONFIG_RT_GROUP_SCHED.

.

This uses the cgroup virtual file system and "<cgroup>/cpu.rt_runtime_us” to control the CPU time reserved for each control group.

It uses the cgroup virtual file system and uses "<cgroup> /cpu.rt_runtime_us " to control CPU time reservations for each control group. .

For more information on working with control groups, you should read Documentation/admin-guide/cgroup-v1/cgroups.rst as well.

Also read Documentation / admin-guide / cgroup-v1 / cgroups.rst for more information on working with control groups.

.

Group settings are checked against the following limits in order to keep the configuration schedulable:

Group settings are checked against the following limits to keep them configurable.

Sum_{i} runtime_{i} / global_period <= global_runtime / global_period

For now, this can be simplified to just the following (but see Future plans):

At the moment, this can be simplified as follows (but see future plans):

Sum_{i} runtime_{i} <= global_runtime

  1. Future plans

There is work in progress to make the scheduling period for each group (“/cpu.rt_period_us”) configurable as well.

Work is also underway to make the scheduling period for each group (“ /cpu.rt_period_us”) configurable.

The constraint on the period is that a subgroup must have a smaller or equal period to its parent.

The time constraint is that the subgroup must have a shorter or equal time period than its parent.

But realistically its not very useful yet as its prone to starvation without deadline scheduling.

However, in reality, without deadline scheduling, it tends to be exhausted and is not useful. .

Consider two sibling groups A and B; both have 50% bandwidth, but A’s period is twice the length of B’s.

Consider two related groups, A and B. Both have 50% bandwidth, but the duration of A is twice that of B.

  • group A: period=100000us, runtime=50000us

This means that currently a while (1) loop in A will run for the full period of B and can starve B’s tasks (assuming they are of lower priority) for a whole period.

This means that the while (1) loop in A runs across all of B's institutions, and B's tasks (assuming low priority) are exhausted over the entire period.

.

The next project will be SCHED_EDF (Earliest Deadline First scheduling) to bring full deadline scheduling to the linux kernel.

The next project is SCHED_EDF (Earliest Deadline First scheduling), which brings complete deadline scheduling to the Linux kernel.

Deadline scheduling the above groups and treating end of the period as a deadline will ensure that they both get their allocated time.

The above group and deadline scheduling, which treats it as the end of the deadline period, ensure that both have their allocated time.

.

Implementing SCHED_EDF might take a while to complete.

The implementation of SCHED_EDF is likely to take some time to complete.

Priority Inheritance is the biggest challenge as the current linux PI infrastructure is geared towards the limited static priority levels 0-99.

In today's Linux PI infrastructure, priority inheritance is the biggest challenge, as static priority levels are limited to 0-99.

With deadline scheduling you need to do deadline inheritance

In deadline scheduling, you must inherit deadline.

(since priority is inversely proportional to the deadline delta (deadline - now)).

(Because the priority is inversely proportional to deadline delta (deadline --now))

.

This means the whole PI machinery will have to be reworked - and that is one of the most complex pieces of code we have.

This means that the entire PI machinery needs to be rebuilt. This is part of the most complex part of the code we have.

Recommended Posts

Real-Time group scheduling