Scroll to navigation

CPUPLUGD.CONF(5) File Formats Manual CPUPLUGD.CONF(5)

NAME

cpuplugd.conf - Configuration file for the Linux on System z CPU and memory hotplug daemon

DESCRIPTION

The cpuplugd.conf configuration file contains the configuration information for the Linux for System z CPU and memory hotplug daemon.

Use this file to specify rules for enabling or disabling CPUs and for adding or removing memory.

CPU hotplug

CPUs can be enabled and disabled through a sysfs interface. The status file for a CPU, here CPU number 16 (counting starts at 0), is /sys/devices/system/cpu/cpu15/online.

Writing a 0 to this file disables the CPU. Writing a 1 enables the CPU.

Memory hotplug

The rules that add or remove memory use the cooperative memory management (CMM) feature.

CMM is a mechanism to reduce the memory available to Linux instances that run as guests of z/VM. CMM allocates pages to a dynamic page pool not available to Linux. A diagnose code indicates to z/VM that the pages in the page pool are out of use. z/VM can then immediately reuse these pages for other guests.

Layout of the configuration file

The configuration file contains variables specifying static numbers or expressions. They are of the format <variable>="<value>" and they need to be specified within one line. Expressions can be specified to calculate algebraic values or to define boolean rules, which determine when a hotplug/hotunplug action should be taken. The maximum valid line length is 2048 characters.

There are case-insensitive pre-defined and case-sensitive user-defined variables. The configuration file must include specifications for all pre-defined variables. If a variable is not set, the hotplug function it applies to, CPU or memory, is disabled. The only exception to this rule is CMM_DEC, which defaults to the setting for CMM_INC if omitted. If a pre-defined variable is set more than once, only the last occurrence is used. User-defined variables must not be set more than once.

Hotplug rules

Set these pre-defined variables to an expression that resolves to a boolean value (true or false). These variables trigger hotplug actions. Setting a variable to "0" disables the action.

-
HOTPLUG - used to enable CPUs
-
HOTUNPLUG - used to disable CPUs
-
MEMPLUG - used to increase the available memory
-
MEMUNPLUG - used to decrease the amount of memory

The following operators can be used in a hotplug rule expression:

+ * ( ) / - < >

Furthermore, the boolean operators & (and) | (or) and ! (not) can be used.

If both HOTPLUG and HOTUNPLUG evaluate to true, only the HOTPLUG action is triggered. If both MEMPLUG and MEMUNPLUG evaluate to true, only the MEMPLUG action is triggered.

Pre-defined static variables

The following pre-defined variables can be set only to a static, positive, numeric value:

-
CPU_MIN - the minimum number of CPUs to keep online (> 0)
-
CPU_MAX - the maximum number of CPUs to enable (>= 0)
-
UPDATE - the interval at which cpuplugd evaluates the rules (in seconds, > 0)
-
CMM_MIN - the minimum size of the CMM page pool (>= 0)
-
CMM_MAX - the maximum size of the CMM page pool (>= 0)

If the value of CPU_MAX is 0, the overall number of CPUs found in this system is used as the maximum.

Pre-defined dynamic variables

The following pre-defined variables can either be set to a static value or to an algebraic expression:

-
CMM_INC - the amount of pages by which the CMM page pool is increased if the MEMUNPLUG rule is matched (available system memory is decreased).
-
CMM_DEC - the amount of pages by which the CMM page pool is decreased if the MEMPLUG rule is matched (available system memory is increased).

The following operators can be used in a dynamic variable expression:

+ * ( ) / - < >

User-defined variables

You can specify complex calculations as user-defined variables, which can then be used in expressions. User-defined variables are case-sensitive and must not match a pre-defined variable or keyword. In the configuration file, definitions for user-defined variables must precede their use in expressions.

Variable names consist of alphanumeric characters (a-z,A-Z,0-9) and the "_" character, see section "EXAMPLES" for an example (pgscanrate). The maximum name length for a variable is 128 characters, and the maximum total size for all user-defined variables (names + values) is 4096 characters.

Keywords for CPU hotplug rules

The HOTPLUG and HOTUNPLUG rules can contain the following pre-defined keywords:

-
loadavg - the current load average
-
onumcpus - the current number of CPUs which are online
-
runnable_proc - the current amount of runnable processes
-
user - the current CPU user percentage
-
nice - the current CPU nice percentage
-
system - the current CPU system percentage
-
idle - the current CPU idle percentage
-
iowait - the current CPU iowait percentage
-
irq - the current CPU irq percentage
-
softirq - the current CPU softirq percentage
-
steal - the current CPU steal percentage
-
guest - the current CPU guest percentage (depends on kernel version: if not reported in /proc/stat, this is set to 0)
-
guest_nice - the current CPU guest_nice percentage (depends on kernel version: if not reported in /proc/stat, this is set to 0)
-
cpustat.<name> - data from /proc/stat and /proc/loadavg
-
time - floating point timestamp in "seconds.microseconds" since the Unix Epoch (1970-01-01 00:00:00 +0000 (UTC))

The percentage values are accumulated over all online CPUs, so they can vary between 0 and (100 * onumcpus).

CPU usage data from /proc/stat and /proc/loadavg is accessible by specifying cpustat.<name>, where <name> can be any of the keywords described above, plus total_ticks. In this case, loadavg, onumcpus and runnable_proc provide the same values as the pre-defined keywords, while the others refer to the raw timer ticks as reported by /proc/stat, not the percentage. For example, cpustat.idle reports the timer ticks spent in idle since system start, and cpustat.total_ticks indicates the sum of all reported timer ticks, which can be useful for user-defined percentage calculations.

Keywords for memory hotplug rules

The MEMPLUG and MEMUNPLUG rules can contain the following pre-defined keywords:

-
apcr - the amount of page cache operations, i.e. pgpin + pgpout from /proc/vmstat (in 512 byte blocks / second)
-
freemem - the amount of free memory (in megabytes)
-
swaprate - the number of swap operations, i.e. pswpin + pswpout from /proc/vmstat (in pages / second)
-
meminfo.<name> - any value from /proc/meminfo
-
vmstat.<name> - any value from /proc/vmstat
-
time - floating point timestamp in "seconds.microseconds" since the Unix Epoch (1970-01-01 00:00:00 +0000 (UTC))

All values from /proc/meminfo and /proc/vmstat can be used in an expression by specifying meminfo.<name> or vmstat.<name>, where <name> matches a symbol name reported by /proc/meminfo or /proc/vmstat (case sensitive), e.g. meminfo.MemTotal.

History function

There is a history function for the following keywords:

-
cpustat.<name> - data from /proc/stat and /proc/loadavg
-
meminfo.<name> - any value from /proc/meminfo
-
vmstat.<name> - any value from /proc/vmstat
-
time - floating point timestamp in "seconds.microseconds" since the Unix Epoch (1970-01-01 00:00:00 +0000 (UTC))

The history levels can be accessed by appending [<history level>] to the name, where <history level> indicates the amount of past intervals where the value was gathered. [0] means the current interval (the [0] can be omitted in this case), [1] means the previous interval, [2] means two intervals ago, and so on. The history limit is 100. For example, cpustat.system[1] would indicate the system value from /proc/stat at the previous interval, while vmstat.pgpgin and vmstat.pgpgin[0] would both mean the current pgpgin value from /proc/vmstat.

The time keyword and its history values can be used to calculate values dependent on time intervals, see section "EXAMPLES" for an example (pgscanrate).

EXAMPLES

A complete configuration file could look like this:

------------------------------ config file start ------------------------------
UPDATE="5"
CPU_MIN="2"
CPU_MAX="5"
CMM_MIN="0"
CMM_MAX="131072"	# 512 MB
pgscan_k="vmstat.pgscan_kswapd_dma + vmstat.pgscan_kswapd_normal + vmstat.pgscan_kswapd_movable"
pgscan_d="vmstat.pgscan_direct_dma + vmstat.pgscan_direct_normal + vmstat.pgscan_direct_movable"
pgscan_k1="vmstat.pgscan_kswapd_dma[1] + vmstat.pgscan_kswapd_normal[1] + vmstat.pgscan_kswapd_movable[1]"
pgscan_d1="vmstat.pgscan_direct_dma[1] + vmstat.pgscan_direct_normal[1] + vmstat.pgscan_direct_movable[1]"
pgscanrate="(pgscan_k + pgscan_d - pgscan_k1 - pgscan_d1) / (time - time[1])"
cache="meminfo.Cached + meminfo.Buffers"
# CMM_INC: 10% of free memory + cache, in 4K pages
CMM_INC="(meminfo.MemFree + cache) / 40"
# CMM_DEC: 10% of total memory in 4K pages
CMM_DEC="meminfo.MemTotal / 40"
HOTPLUG = "(loadavg > onumcpus + 0.75) & (idle < 10.0)"
HOTUNPLUG = "(loadavg < onumcpus - 0.25) | (idle > 50)"
# Plug memory if page scan rate is above 20 pages / sec
MEMPLUG = "pgscanrate > 20"
# Unplug memory while free memory is above 10% of total memory, or cache uses
# more than 50% of total memory
MEMUNPLUG = "(meminfo.MemFree > meminfo.MemTotal / 10) | (cache > meminfo.MemTotal / 2)"
------------------------------ config file end ------------------------------

The example includes multiple user-defined variables to calculate the page scan rate with values from /proc/vmstat, as well as the cache size.

Attention: Do not use these example rules on production systems. The rules have been designed to illustrate the configuration file syntax and are not suitable for actually governing hotplug actions. Useful rules differ considerably depending on the workload, resources, and requirements of the system they are designed for.

SEE ALSO

cpuplugd(8)

May 2011 s390-tools