Edmondo's Vault

Linux cgroups

May 18, 20263 min read

Cgroups are a way to organize process gruops hierarchically and control system resource usage such as RAM or CPU. There are two versions of cgroups available in the kernel: cgroups v1 and v2. cgroup v1 is still widely used, but v2 will eventually replace v1, so you should focus on v2.

cgroup v1

With cgroup v1, the community had an ad hoc approach, adding new cgroups and controllers as needed. The following v1 cgroups and controllers exist (ordered from oldest to newest; note that the docs are all over the place and inconsistent):

  • CFS bandwidth control. Used via the cpu cgroup. Supported since Linux 2.6.24.
  • CPU accounting controller. Used via the cpuacct cgroup. Supported since Linux 2.6.24.
  • cpusets cgroup. Allows you to assign CPU and memory to a task. Supported since Linux 2.6.24.
  • Memory resource controller. Allows you to isolate the memory behavior of tasks. Supported since Linux 2.6.25.
  • Device whitelist controller. Allows you to control device file usage. Supported since Linux 2.6.26.
  • freezer cgroup Used for batch job management. Supported since Linux 2.6.28.
  • Network classifier cgroup Used to assign different priorities to packets. Supported since Linux 2.6.29.
  • Block IO controller Allows you to throttle block I/O. Supported since Linux 2.6.33.
  • perf_event command Allows you to collect performance data. Supported since Linux 2.6.39.
  • Network priority cgroup Allows you to dynamically set the priority of network traffic. Supported since Linux 3.3.
  • HugeTLB controller Allows you to limit HugeTLB usage. Supported since Linux 3.5.
  • Process number controller Used to allow a cgroup hierarchy to stop creating new processes after a certain limit is reached. Supported since Linux 4.3.

cgroups v2

cgroup v2 is a total rewrite of cgroups with the lessons learned from v1. This is true both in terms of consistent configuration and use of the cgroups as well as the (centralized and uniform) documentation. Unlike the per-process cgroup v1 design, cgroup v2 has only single hierarchy, and all controllers are managed the same way. Here are the v2 controllers:

  • CPU controller. Regulates distribution of CPU cycles, supporting different models (weight, max) and includes usage reporting
  • Memory controller. Regulates distribution of memory with a range of control parameters, supporting user-space memory, kernel data structures such as dentries and inodes, and TCP socket buffers
  • I/O controller. Regulates the distribution of I/O resources with both weight-based and absolute bandwidth or I/O operations per second (IOPS) limits, reporting on bytes and IOPS read/writes
  • Process number (PID) controller. Is similar to the v1 version
  • cpuset controller. Is similar to the v1 version
  • device controller. Manages access to device files, implemented on top of eBPF
  • rdma controller. Regulates the distribution and accounting of remote direct memory access (RDMA) resources
  • HugeTLB controller Is similar to the v1 version

There are also miscellaneous cgroups in v2 that allow resource limits and tracking mechanisms for scalar resources (which can’t be abstracted like other cgroup resources).

Tip

You can view alll of the v2 cgroups in Linux in tree format using systemctl status and interactive resource usage using systemd-cgtop, while cinf can be used to list namespaces and cgroups

Graph View

Backlinks