system76-scheduler sweeps your processes and rewrites the CFS latencies

Two knobs, and the daemon turns them for you

The service does two things, and it is worth separating them because they have different risk profiles. The first is the CPU scheduler configuration for the Completely Fair Scheduler, the kernel scheduler that decides how quickly a runnable task gets the CPU. The daemon sets the low latency CFS scheduling automatically when the machine is on AC power, and uses the default scheduling latencies on battery. The second is per process priority, and processes are regularly swept and assigned priorities based on the configuration files. Combined with pop-shell, foreground processes and their sub-processes are given higher process priority, which is the mechanism by which the window you are looking at wins time against everything else. The stated effect is that background applications and services get a smaller portion of the leftover CPU budget after the active process has had the most time on the CPU, so the improvement is most visible on older hardware with budget parts, and games benefit from higher frame rates and less jitter.

The configuration is KDL, and the system copy wins

Configuration lives in two places. The system configuration is at /etc/system76-scheduler/config.kdl and the distribution configuration is at /usr/share/system76-scheduler/config.kdl, and the presence of the system configuration overrides the distribution one. The documented default configuration is in the repository at data/config.kdl. The format is KDL, a structured document language, and two of its nodes have a special meaning for what the daemon does. If the background and foreground assignment profiles are defined, foreground process management is enabled, which is the switch for the pop-shell integration. And if a pipewire profile is defined, pipewire process monitoring is enabled, so audio applications get their own treatment. That coupling is a design decision worth understanding before you edit anything, because deleting a profile to disable one behaviour can silently disable another.

Process scheduler profiles live in their own directory

Separate from the main configuration file, additional process scheduling profiles are stored in a user configuration directory at /etc/system76-scheduler/process-scheduler/ and a distribution directory at /usr/share/system76-scheduler/process-scheduler/. An example configuration is provided in the repository as data/pop_os.kdl, and the format is parsed the same way as the assignments and exceptions nodes in the main config. One feature of the format is inheritance: profiles can inherit values from the previous assignment of the same name, so a later profile only needs to state what changes. The block itself is small:

assignments {
    {{profile-name}} {{profile-properties}}
}

The profile name can refer to any name of your choice, and if the name matches a previous assignment it inherits the values from that assignment plus any additional profile properties assigned. That is a genuinely useful property for a configuration that grows over time, since a small tweak to one profile does not mean rewriting a long file.

Nice, scheduling policy and I/O priority are the three properties

A profile can set three kinds of property. The first is nice priority, defined as nice=-20 through nice=19, where the usual Linux convention applies and lower numbers mean more CPU for a process that is allowed to have it. The second is a scheduler policy, one of sched="batch", sched="idle", sched="other", or the realtime policies sched=(fifo)1 through sched=(fifo)99 and sched=(rr)1 through sched=(rr)99. The README notes that realtime policies assign a priority level between 1 and 99, that higher values have higher priority, and that it is recommended not to set a priority higher than hardware IRQs, which it puts at greater than 49. The third is an I/O priority, either io="idle", io=(best-effort)0 through io=(best-effort)7, or io=(realtime)0 through io=(realtime)7, where the best-effort and realtime classes run from 0 to 7, with 7 the least priority and 0 the highest.

Matching rules are what make the profiles useful

Each child element of a profile defines the processes to assign to it, and the matching is what turns a static list into something that survives a reboot:

{{profile-name}} {{profile-properties}} {
    "/match/by/cmdline" {{profile-properties}}
    match-by-name {{profile-properties}}
    * {{condition-properties}} {{profile-properties}}
}

A node name starting with a slash is a match by command line path, a node name otherwise is a match by process name, and the star matches all processes and is used with additional condition properties. Those conditions are wild-match'd, and a property may start with an exclamation mark to exclude the results that match the condition. Two named conditions matter for containers and for process trees: cgroup="cgroup-path" matches processes by a cgroup, and parent="name" matches processes by the process name of the parent. Matching by cgroup is the interesting one for anyone running services under systemd, since it lets a whole unit be treated as a unit rather than a list of binary paths.

The CFS latencies are two short blocks of YAML

The scheduler side is configured by four numbers, and the README gives two sets of them. The default is the Linux kernel default for CFS, which achieves a high level of throughput for CPU bound tasks at the cost of increased latency for inputs, and the README calls it ideal for servers and laptops on battery, since low latency scheduling sacrifices some energy efficiency for improved responsiveness:

latency: 6ns
minimum_granularity: 0.75ms
wakeup_granularity: 1.0ms
bandwidth_size: 5us

The responsive setting slightly reduces the time given to CPU bound tasks to give more time to other processes, particularly those awaiting and responding to user inputs, which can significantly improve desktop responsiveness for a slight penalty in throughput:

latency: 4ns
minimum_granularity: 0.4ms
wakeup_granularity: 0.5ms
bandwidth_size: 3us

The daemon switches between these based on power state, which is the whole point of the design: the latency cost is paid when the machine is plugged in and not paid when it is not.

Building it needs a tracer, a justfile and a systemd install

Installation from source is documented in the README and is unusual in one respect. The dependencies are the ones defined in the debian/control file: cargo and rustc, clang, just, libclang-dev, libpipewire-0.3-dev and pkg-config. Then the included justfile builds and installs:

just execsnoop=$(which execsnoop-bpfcc) build-release
sudo just sysconfdir=/usr/share install

The execsnoop argument is the clue to how the daemon works. It needs an execsnoop tracer, and the justfile resolves it by looking for execsnoop, then execsnoop-bpfcc, and falling back to /usr/sbin/execsnoop-bpfcc, which is how the daemon learns about newly started processes. The justfile also switches the linker to lld when it is available, passes the tracer path through the EXECSNOOP_PATH environment variable, offers a vendored build for offline work, runs clippy with the pedantic lint set for checks, and installs the configuration files, the binary and the systemd unit. The Rust code is a Cargo workspace with four members, config, daemon, execsnoop and pipewire, so the tracer, the configuration parser and the pipewire monitoring are separate crates.

The D-Bus interface is how a desktop tells it what is focused

The daemon exposes a D-Bus interface named com.system76.Scheduler at the path /com/system76/Scheduler, and the SetForegroundProcess(u32) method can be called to change the active foreground process. That is the integration point with pop-shell, and it is why the daemon does not have to guess which window has focus. The alternative to this design is instructive. Without a D-Bus hook, a scheduler daemon has to infer the foreground process by reading the active window from the display server and mapping it back to a process, which is fragile and depends on the desktop environment. With the hook, the desktop tells the scheduler directly, and the scheduler applies the configured foreground profile to that process and its children. The trade is that the daemon is coupled to a specific desktop, which is why the pop-shell combination is called out in the README rather than treated as one option among many.

Editorial conclusion

Adopt system76-scheduler if you run a desktop where background work competes with what you are doing, and you want the priorities set automatically and adjusted for whether the machine is on mains power. Do not adopt it as a general purpose tuning tool for servers, since the design targets desktop responsiveness and the daemon is a privileged service that sweeps processes on a schedule. Before you install it, read the default configuration in data/config.kdl and the example profile, check that the execsnoop tracer from bpfcc is available on your kernel, and decide whether you want the scheduler latencies changed at all, because that is a separate decision from the per process priorities.

Frequently asked questions

Official sources

1. Issues
2. License: MPL-2.0
3. pop-os/system76-scheduler on GitHub
4. README
5. Releases