CTDBD.CONF(5) | CTDB - clustered TDB database | CTDBD.CONF(5) |
NAME¶
ctdbd.conf - CTDB daemon configuration file
DESCRIPTION¶
This file contains CTDB configuration variables that are affect the operation of CTDB. The default location of this file is /etc/ctdb/ctdbd.conf.
This file is a shell script (see sh(1)) but is usually limited to simple variable assignments and shell-style comments.
CTDB configuration variables are grouped into several categories below.
Variables defined in this document can also be set in a distribution-specific configuration file such as /etc/sysconfig/ctdb (Red Hat) or /etc/default/ctdb (Debian). However, these files should be reserved for variables used by the initscript. A historical alternative is /etc/ctdb/sysconfig/ctdb - this is deprecated.
INITSCRIPT CONFIGURATION¶
Some options must be available to the initscript so they need to be set in the distribution-specific initscript configuration, such as /etc/sysconfig/ctdb or /etc/default/ctdb.
CTDB_PIDFILE=FILENAME
Default is /var/run/ctdb/ctdbd.pid. Corresponds to --pidfile.
GLOBAL CONFIGURATION¶
These options may be used in the initscripts, daemon and scripts.
CTDB_BASE=DIRECTORY
DAEMON CONFIGURATION¶
Variables in this section are processed by ctdbd_wrapper(1) and are converted into command-line arguments to ctdbd(1). Correspondence with ctdbd(1) options is shown for each variable. The the documentation for the relevant options for more details.
Many of these variables are also used by event scripts.
CTDB_CAPABILITY_LMASTER=yes|no
CTDB_CAPABILITY_RECMASTER=yes|no
CTDB_DBDIR=DIRECTORY
Apart from a DIRECTORY, this can take a special value of the form tmpfs[:OPTIONS]. OPTIONS is a comma-separated list of any permissible options to the tmpfs filesystem. The only pre-specified default is mode=700, which can overridden by specifying mode in OPTIONS. It probably makes sense to specify a maximum size.
Corresponds to --dbdir.
CTDB_DBDIR_PERSISTENT=DIRECTORY
Corresponds to --dbdir-persistent.
CTDB_DBDIR_STATE=DIRECTORY
Corresponds to --dbdir-state.
CTDB_DEBUGLEVEL=DEBUGLEVEL
CTDB_EVENT_SCRIPT_DIR=DIRECTORY
CTDB_LOGGING=STRING
Valid values are:
file:FILENAME
syslog[:METHOD]
If METHOD is specified then it specifies an extension that causes logging to be done in a non-blocking fashion. This can be useful under heavy loads that might cause the syslog daemon to dequeue messages too slowly, which would otherwise cause CTDB to block when logging. METHOD must be one of:
nonblocking
udp
udp-rfc5424
CTDB_NODES=FILENAME
CTDB_NOSETSCHED=yes|no
Usually CTDB runs with real-time priority. If you are running CTDB on a platform that does not support real-time priority, you can set this.
CTDB_NOTIFY_SCRIPT=FILENAME
CTDB_MAX_PERSISTENT_CHECK_ERRORS=NUM
CTDB_NODE_ADDRESS=IPADDR
By default ctdbd will select the first address from the nodes list that in can bind to. See also CTDB_NODES.
This option is only required when automatic address detection can not be used. This can be the case when running multiple ctdbd daemons/nodes on the same physical host (usually for testing), using InfiniBand for the private network or on Linux when sysctl net.ipv4.ip_nonlocal_bind=1.
CTDB_PUBLIC_ADDRESSES=FILENAME
CTDB_PUBLIC_INTERFACE=INTERFACE
CTDB_RECOVERY_LOCK=LOCK
No default, but the default configuration file specifies /some/place/on/shared/storage, which should be change to a useful value. Corresponds to --reclock.
For information about the recovery lock please see the RECOVERY LOCK section in ctdb(7).
CTDB_SCRIPT_LOG_LEVEL=DEBUGLEVEL
CTDB_SOCKET=FILENAME
If you change this then you probably want to set this in root's environment (perhaps in a file in /etc/profile.d) so that you can use the ctdb(1) command in a straightforward manner.
CTDB_START_AS_DISABLED=yes|no
CTDB_START_AS_STOPPED=yes|no
CTDB_TRANSPORT=tcp|infiniband
While the following variables do not translate into daemon options they are used by ctdbd_wrapper(1) when starting and stopping ctdbd(1).
CTDB_SHUTDOWN_TIMEOUT=NUM
Defaults is 30.
CTDB_STARTUP_TIMEOUT=NUM
Defaults is 10.
NETWORK CONFIGURATION¶
NAT GATEWAY¶
NAT gateway is used to configure fallback routing for nodes when they do not host any public IP addresses. For example, it allows unhealthy nodes to reliably communicate with external infrastructure. One node in a NAT gateway group will be designated as the NAT gateway master node and other (slave) nodes will be configured with fallback routes via the NAT gateway master node. For more information, see the NAT GATEWAY section in ctdb(7).
CTDB_NATGW_DEFAULT_GATEWAY=IPADDR
No default. Setting this variable is optional - if not set that no route is created on the NAT gateway master node.
CTDB_NATGW_NODES=FILENAME
File format:
IPADDR [slave-only]
IPADDR is the private IP address of each node in the NAT gateway group.
If "slave-only" is specified then the corresponding node can not be the NAT gateway master node. In this case CTDB_NATGW_PUBLIC_IFACE and CTDB_NATGW_PUBLIC_IP are optional and unused.
No default, usually /etc/ctdb/natgw_nodes when enabled.
CTDB_NATGW_PRIVATE_NETWORK=IPADDR/MASK
No default.
CTDB_NATGW_PUBLIC_IFACE=IFACE
No default.
CTDB_NATGW_PUBLIC_IP=IPADDR/MASK
No default.
CTDB_NATGW_STATIC_ROUTES=IPADDR/MASK[@GATEWAY] ...
If GATEWAY is specified then the corresponding route on the NATGW master node will be via GATEWAY. Such routes are created even if CTDB_NATGW_DEFAULT_GATEWAY is not specified. If GATEWAY is not specified for some networks then routes are only created on the NATGW master node for those networks if CTDB_NATGW_DEFAULT_GATEWAY is specified.
This should be used with care to avoid causing traffic to unnecessarily double-hop through the NAT gateway master, even when a node is hosting public IP addresses. Each specified network or host should probably have a corresponding automatically created link route or static route to avoid this.
No default.
Example
CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24 CTDB_NATGW_DEFAULT_GATEWAY=10.0.0.1 CTDB_NATGW_PUBLIC_IP=10.0.0.227/24 CTDB_NATGW_PUBLIC_IFACE=eth0
A variation that ensures that infrastructure (ADS, DNS, ...) directly attached to the public network (10.0.0.0/24) is always reachable would look like this:
CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24 CTDB_NATGW_PUBLIC_IP=10.0.0.227/24 CTDB_NATGW_PUBLIC_IFACE=eth0 CTDB_NATGW_STATIC_ROUTES=10.0.0.0/24
Note that CTDB_NATGW_DEFAULT_GATEWAY is not specified.
POLICY ROUTING¶
A node running CTDB may be a component of a complex network topology. In particular, public addresses may be spread across several different networks (or VLANs) and it may not be possible to route packets from these public addresses via the system's default route. Therefore, CTDB has support for policy routing via the 13.per_ip_routing eventscript. This allows routing to be specified for packets sourced from each public address. The routes are added and removed as CTDB moves public addresses between nodes.
For more information, see the POLICY ROUTING section in ctdb(7).
CTDB_PER_IP_ROUTING_CONF=FILENAME
The special FILENAME value __auto_link_local__ indicates that no configuration file is provided and that CTDB should generate reasonable link-local routes for each public IP address.
File format:
IPADDR DEST-IPADDR/MASK [GATEWAY-IPADDR]
No default, usually /etc/ctdb/policy_routing when enabled.
CTDB_PER_IP_ROUTING_RULE_PREF=NUM
This should be (strictly) greater than 0 and (strictly) less than 32766. A priority of 100 is recommended, unless this conflicts with a priority already in use on the system. See ip(8), for more details.
CTDB_PER_IP_ROUTING_TABLE_ID_LOW=LOW-NUM, CTDB_PER_IP_ROUTING_TABLE_ID_HIGH=HIGH-NUM
ip(8) uses some reserved routing table numbers below 255. Therefore, CTDB_PER_IP_ROUTING_TABLE_ID_LOW should be (strictly) greater than 255.
CTDB uses the standard file /etc/iproute2/rt_tables to maintain a mapping between the routing table numbers and labels. The label for a public address ADDR will look like ctdb.addr. This means that the associated rules and routes are easy to read (and manipulate).
No default, usually 1000 and 9000.
Example
CTDB_PER_IP_ROUTING_CONF=/etc/ctdb/policy_routing CTDB_PER_IP_ROUTING_RULE_PREF=100 CTDB_PER_IP_ROUTING_TABLE_ID_LOW=1000 CTDB_PER_IP_ROUTING_TABLE_ID_HIGH=9000
LVS¶
For a general description see the LVS section in ctdb(7).
Eventscript
CTDB_LVS_NODES=FILENAME
File format:
IPADDR [slave-only]
IPADDR is the private IP address of each node in the LVS group.
If "slave-only" is specified then the corresponding node can not be the LVS master node. In this case CTDB_LVS_PUBLIC_IFACE and CTDB_LVS_PUBLIC_IP are optional and unused.
No default, usually /etc/ctdb/lvs_nodes when enabled.
CTDB_LVS_PUBLIC_IFACE=INTERFACE
CTDB_LVS_PUBLIC_IP=IPADDR
MISCELLANEOUS NETWORK CONFIGURATION¶
CTDB_PARTIALLY_ONLINE_INTERFACES=yes|no
Note that CTDB_PARTIALLY_ONLINE_INTERFACES=yes is incompatible with NAT gateway, since NAT gateway relies on the interface configured by CTDB_NATGW_PUBLIC_IFACE to be up.
Default is "no".
SERVICE CONFIGURATION¶
CTDB can be configured to manage and/or monitor various NAS (and other) services via its eventscripts.
In the simplest case CTDB will manage a service. This means the service will be started and stopped along with CTDB, CTDB will monitor the service and CTDB will do any required reconfiguration of the service when public IP addresses are failed over.
SAMBA¶
Eventscripts
CTDB_MANAGES_SAMBA=yes|no
Default is no.
CTDB_MANAGES_WINBIND=yes|no
Default is no.
CTDB_SAMBA_CHECK_PORTS=PORT-LIST
Default is to monitor ports that Samba is configured to listen on.
CTDB_SAMBA_SKIP_SHARE_CHECK=yes|no
Default is no.
CTDB_SERVICE_NMB=SERVICE
Default is distribution-dependant.
CTDB_SERVICE_SMB=SERVICE
Default is distribution-dependant.
CTDB_SERVICE_WINBIND=SERVICE
Default is "winbind".
NFS¶
This includes parameters for the kernel NFS server. Alternative NFS subsystems (such as NFS-Ganesha[1]) can be integrated using CTDB_NFS_CALLOUT.
Eventscript
CTDB_MANAGES_NFS=yes|no
Default is no.
CTDB_NFS_CALLOUT=COMMAND
Default is the included nfs-linux-kernel-callout.
CTDB_NFS_CHECKS_DIR=DIRECTORY
CTDB_NFS_CHECKS_DIR can be used to point to different sets of checks for different NFS servers.
One way of using this is to have it point to, say, /etc/ctdb/nfs-checks-enabled.d and populate it with symbolic links to the desired check files. This avoids duplication and is upgrade-safe.
Default is /etc/ctdb/nfs-checks.d, which contains NFS RPC checks suitable for Linux kernel NFS.
CTDB_NFS_SKIP_SHARE_CHECK=yes|no
Default is no.
CTDB_RPCINFO_LOCALHOST=IPADDR|HOSTNAME
Default is "127.0.0.1".
CTDB_RPCINFO_LOCALHOST6=IPADDR|HOSTNAME
Default is "::1".
CTDB_NFS_STATE_FS_TYPE=TYPE
CTDB_NFS_STATE_MNT=DIR
APACHE HTTPD¶
CTDB can manage the Apache web server.
Eventscript
CTDB_MANAGES_HTTPD=yes|no
Default is no.
CLAMAV¶
CTDB has support to manage the popular anti-virus daemon ClamAV.
Eventscript
This eventscript is not enabled by default. Use ctdb enablescript to enable it.
CTDB_MANAGES_CLAMD=yes|no
Default is no.
CTDB_CLAMD_SOCKET=FILENAME
No default.
ISCSI¶
CTDB has support for managing the Linux iSCSI tgtd service.
Eventscript
CTDB_MANAGES_ISCSI=yes|no
Default is no.
CTDB_START_ISCSI_SCRIPTS=DIRECTORY
No default.
MULTIPATHD¶
CTDB can monitor multipath devices to ensure that active paths are available.
Eventscript
This eventscript is not enabled by default. Use ctdb enablescript to enable it.
CTDB_MONITOR_MPDEVICES=MP-DEVICE-LIST
No default.
VSFTPD¶
CTDB can manage the vsftpd FTP server.
Eventscript
CTDB_MANAGES_VSFTPD=yes|no
Default is no.
SYSTEM RESOURCE MONITORING CONFIGURATION¶
CTDB can experience seemingly random (performance and other) issues if system resources become too constrained. Options in this section can be enabled to allow certain system resources to be checked. They allows warnings to be logged and nodes to be marked unhealthy when system resource usage reaches the configured thresholds.
Some checks are enabled by default. It is recommended that these checks remain enabled or are augmented by extra checks. There is no supported way of completely disabling the checks.
Eventscripts
Filesystem and memory usage monitoring is in 05.system.
CTDB_MONITOR_FILESYSTEM_USAGE=FS-LIMIT-LIST
Default is to warn for each filesystem containing a database directory (CTDB_DBDIR, CTDB_DBDIR_PERSISTENT, CTDB_DBDIR_STATE) with a threshold of 90%.
CTDB_MONITOR_MEMORY_USAGE=MEM-LIMITS
Default is 80, so warnings will be logged when memory usage reaches 80%.
CTDB_MONITOR_SWAP_USAGE=SWAP-LIMITS
Default is 25, so warnings will be logged when swap usage reaches 25%.
MISCELLANEOUS SERVICE-RELATED CONFIGURATION¶
CTDB_MANAGED_SERVICES=SERVICE-LIST
No default.
TUNABLES CONFIGURATION¶
CTDB tunables (see ctdbd-tunables(7)) can be set from the configuration file. They are set as follows:
CTDB_SET_TUNABLE=VALUE
For example:
CTDB_SET_MonitorInterval=20
DEBUG AND TEST¶
Variable in this section are for debugging and testing CTDB. They should not generally be needed.
CTDB_DEBUG_HUNG_SCRIPT=FILENAME
Default is CTDB_BASE/debug-hung-script.sh.
CTDB_DEBUG_HUNG_SCRIPT_LOGFILE=FILENAME
No default. Messages go to stdout/stderr and are logged to the same place as other CTDB log messages.
CTDB_DEBUG_HUNG_SCRIPT_STACKPAT=REGEXP
Default is "exportfs|rpcinfo".
CTDB_DEBUG_LOCKS=FILENAME
No default, usually CTDB_BASE/debug_locks.sh.
CTDB_SYS_ETCDIR=DIRECTORY
Default is /etc.
CTDB_INIT_STYLE=debian|redhat|suse
No fixed default.
If this option needs to be changed from the calculated default for the initscript to function properly, then it must be set in the distribution-specific initscript configuration, such as /etc/sysconfig/ctdb
CTDB_MAX_CORRUPT_DB_BACKUPS=NUM
Default is 10.
CTDB_MAX_OPEN_FILES=NUM
There is no default.
CTDB_RC_LOCAL=FILENAME
No default.
CTDB_RUN_TIMEOUT_MONITOR=yes|no
Default is no.
CTDB_SUPPRESS_COREFILE=yes|no
Default is no.
CTDB_VALGRIND=yes|no|COMMAND
Default is no.
CTDB_VARDIR=DIRECTORY
Defaults to /var/lib/ctdb.
FILES¶
SEE ALSO¶
ctdbd(1), ctdbd_wrapper(1), onnode(1), ctdb(7), ctdb-tunables(7), http://ctdb.samba.org/
AUTHOR¶
This documentation was written by Amitay Isaacs, Martin Schwenke
COPYRIGHT¶
Copyright © 2007 Andrew Tridgell, Ronnie Sahlberg
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, see http://www.gnu.org/licenses.
NOTES¶
- 1.
- NFS-Ganesha
08/30/2018 | ctdb |