Scroll to navigation

CDIST(1) cdist CDIST(1)

NAME

cdist - Usable Configuration Management

SYNOPSIS

cdist [-h] [-V] {banner,config,install,inventory,preos,shell,info} ...
cdist banner [-h] [-l LOGLEVEL] [-q] [-v]
cdist config [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]


             [-g CONFIG_FILE] [-4] [-6] [-C CACHE_PATH_PATTERN]


             [-c CONF_DIR] [-i MANIFEST] [-j [JOBS]] [--log-server]


             [-n] [-o OUT_PATH] [-P] [-R [{tar,tgz,tbz2,txz}]]


             [-r REMOTE_OUT_PATH] [--remote-copy REMOTE_COPY]


             [--remote-exec REMOTE_EXEC] [-S] [-I INVENTORY_DIR] [-A]


             [-a] [-f HOSTFILE] [-p [HOST_MAX]] [-s] [-t]


             [host [host ...]]
cdist install [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]


              [-g CONFIG_FILE] [-4] [-6] [-C CACHE_PATH_PATTERN]


              [-c CONF_DIR] [-i MANIFEST] [-j [JOBS]] [--log-server]


              [-n] [-o OUT_PATH] [-P] [-R [{tar,tgz,tbz2,txz}]]


              [-r REMOTE_OUT_PATH] [--remote-copy REMOTE_COPY]


              [--remote-exec REMOTE_EXEC] [-S] [-I INVENTORY_DIR] [-A]


              [-a] [-f HOSTFILE] [-p [HOST_MAX]] [-s] [-t]


              [host [host ...]]
cdist inventory [-h] {add-host,add-tag,del-host,del-tag,list} ...
cdist inventory add-host [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]


                         [-g CONFIG_FILE] [-I INVENTORY_DIR] [-f HOSTFILE]


                         [host [host ...]]
cdist inventory add-tag [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]


                        [-g CONFIG_FILE] [-I INVENTORY_DIR] [-f HOSTFILE]


                        [-T TAGFILE] [-t TAGLIST]


                        [host [host ...]]
cdist inventory del-host [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]


                         [-g CONFIG_FILE] [-I INVENTORY_DIR] [-a]


                         [-f HOSTFILE]


                         [host [host ...]]
cdist inventory del-tag [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]


                        [-g CONFIG_FILE] [-I INVENTORY_DIR] [-a]


                        [-f HOSTFILE] [-T TAGFILE] [-t TAGLIST]


                        [host [host ...]]
cdist inventory list [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]


                     [-g CONFIG_FILE] [-I INVENTORY_DIR] [-a] [-f HOSTFILE]


                     [-H] [-t]


                     [host [host ...]]
cdist preos [-h] [-l LOGLEVEL] [-q] [-v] [-c CONF_DIR] [-g CONFIG_FILE] [-L]


            [preos] ...
cdist preos [preos-options] debian [-h] [-l LOGLEVEL] [-q] [-v] [-b] [-a ARCH] [-B]


                                   [-C] [-c CDIST_PARAMS] [-D DRIVE] [-e REMOTE_EXEC]


                                   [-i MANIFEST] [-k KEYFILE ] [-m MIRROR]


                                   [-P ROOT_PASSWORD] [-p PXE_BOOT_DIR] [-r]


                                   [-S SCRIPT] [-s SUITE] [-y REMOTE_COPY]


                                   target_dir
cdist preos [preos-options] devuan [-h] [-l LOGLEVEL] [-q] [-v] [-b] [-a ARCH] [-B]


                                   [-C] [-c CDIST_PARAMS] [-D DRIVE] [-e REMOTE_EXEC]


                                   [-i MANIFEST] [-k KEYFILE ] [-m MIRROR]


                                   [-P ROOT_PASSWORD] [-p PXE_BOOT_DIR] [-r]


                                   [-S SCRIPT] [-s SUITE] [-y REMOTE_COPY]


                                   target_dir
cdist preos [preos-options] ubuntu [-h] [-l LOGLEVEL] [-q] [-v] [-b] [-a ARCH] [-B]


                                   [-C] [-c CDIST_PARAMS] [-D DRIVE] [-e REMOTE_EXEC]


                                   [-i MANIFEST] [-k KEYFILE ] [-m MIRROR]


                                   [-P ROOT_PASSWORD] [-p PXE_BOOT_DIR] [-r]


                                   [-S SCRIPT] [-s SUITE] [-y REMOTE_COPY]


                                   target_dir
cdist shell [-h] [-l LOGLEVEL] [-q] [-v] [--colors WHEN] [-s SHELL]
cdist info [-h] [-a] [-c CONF_DIR] [-e] [-F] [-f] [-g CONFIG_FILE] [-t]


           [pattern]
cdist scan -I INTERFACE [--m MODE] [--name-mapper PATH_TO_SCRIPT] [--list]


           [-d CONFIG_DELAY] [-t TRIGGER_DELAY]


DESCRIPTION

cdist is the frontend executable to the cdist configuration management. It supports different subcommands as explained below.

It is written in Python so it requires python(1) to be installed. It requires a minimal Python version 3.5.

GENERAL

All commands accept the following options:

Show the help screen.
Colorize cdist's output. If enabled, cdist will use different colors for different log levels. WHEN recognizes the values 'always', 'never', and 'auto' (the default).

If the value is 'auto', colored output is enabled if stdout is a TTY unless the NO_COLOR (https://no-color.org/) environment variable is defined.

Set the specified verbosity level. The levels, in order from the lowest to the highest, are: ERROR (-1), WARNING (0), INFO (1), VERBOSE (2), DEBUG (3), TRACE (4 or higher). If used along with -v then -v increases last set value and -l overwrites last set value.
Quiet mode: disables logging, including WARNING and ERROR.
Increase the verbosity level. Every instance of -v increments the verbosity level by one. Its default value is 0 which includes ERROR and WARNING levels. The levels, in order from the lowest to the highest, are: ERROR (-1), WARNING (0), INFO (1), VERBOSE (2), DEBUG (3), TRACE (4 or higher). If used along with -l then -l overwrites last set value and -v increases last set value.
Show version and exit.

BANNER

Displays the cdist banner. Useful for printing cdist posters - a must have for every office.

CONFIG/INSTALL

Configure/install one or more hosts. Install command is currently in beta.

-4, --force-ipv4
Force to use IPv4 addresses only. No influence for custom remote commands.
-6, --force-ipv6
Force to use IPv6 addresses only. No influence for custom remote commands.
Use all hosts present in tags db. Currently in beta.
List hosts that have all specified tags, if -t/--tag is specified.
Enable beta functionality.
Specify custom cache path pattern. If it is not set then default hostdir is used. For more info on format see CACHE PATH PATTERN FORMAT below.
Add a configuration directory. Can be specified multiple times. If configuration directories contain conflicting types, explorers or manifests, then the last one found is used.
Read specified file for a list of additional hosts to operate on or if '-' is given, read stdin (one host per line). For the file format see HOSTFILE FORMAT below.
Use specified custom configuration file.
Use specified custom inventory directory. Inventory directory is set up by the following rules: if cdist configuration resolves this value then specified directory is used, if HOME env var is set then ~/.cdit/inventory is used, otherwise distribution inventory directory is used.
Path to a cdist manifest or - to read from stdin.
Operate in parallel in specified maximum number of jobs. Global explorers, object prepare and object run are supported. Without argument CPU count is used by default.
Start a log server for sub processes to use. This is mainly useful when running cdist nested from a code- local script. Log server is always implicitly started for 'install' command.
Do not execute code.
Directory to save cdist output in.
Timestamp log messages with the current local date and time in the format: YYYYMMDDHHMMSS.us.
Operate on multiple hosts in parallel for specified maximum hosts at a time. Without argument CPU count is used by default.
Operate by using archiving with compression where appropriate. Supported values are: tar - tar archive, tgz - gzip tar archive (the default), tbz2 - bzip2 tar archive and txz - lzma tar archive. Currently in beta.
Directory to save cdist output in on the target host.
Disable saving output streams.
Operate on multiple hosts sequentially (default).
Command to use for remote copy (should behave like scp).
Command to use for remote execution (should behave like ssh).
Host is specified by tag, not hostname/address; list all hosts that contain any of specified tags. Currently in beta.

HOSTFILE FORMAT

The HOSTFILE contains one host per line. A comment is started with '#' and continues to the end of the line. Any leading and trailing whitespace on a line is ignored. Empty lines are ignored/skipped.

The Hostfile lines are processed as follows. First, all comments are removed. Then all leading and trailing whitespace characters are stripped. If such a line results in empty line it is ignored/skipped. Otherwise, host string is used.

CACHE PATH PATTERN FORMAT

Cache path pattern specifies path for a cache directory subdirectory. In the path, '%N' will be substituted by the target host, '%h' will be substituted by the calculated host directory, '%P' will be substituted by the current process id. All format codes that python datetime.strftime() function supports, except '%h', are supported. These date/time directives format cdist config/install start time.

If empty pattern is specified then default calculated host directory is used.

Calculated host directory is a hash of a host cdist operates on.

Resulting path is used to specify cache path subdirectory under which current host cache data are saved.

INVENTORY

Manage inventory database. Currently in beta with all sub-commands.

INVENTORY ADD-HOST

Add host(s) to inventory database.

Host(s) to add.
Enable beta functionality.
Read additional hosts to add from specified file or from stdin if '-' (each host on separate line). Hostfile format is the same as config hostfile format.
Use specified custom configuration file.
Use specified custom inventory directory. Inventory directory is set up by the following rules: if cdist configuration resolves this value then specified directory is used, if HOME env var is set then ~/.cdit/inventory is used, otherwise distribution inventory directory is used.

INVENTORY ADD-TAG

Add tag(s) to inventory database.

List of host(s) for which tags are added.
Enable beta functionality.
Read additional hosts to add tags from specified file or from stdin if '-' (each host on separate line). Hostfile format is the same as config hostfile format.
Use specified custom configuration file.
Use specified custom inventory directory. Inventory directory is set up by the following rules: if cdist configuration resolves this value then specified directory is used, if HOME env var is set then ~/.cdit/inventory is used, otherwise distribution inventory directory is used.
Read additional tags to add from specified file or from stdin if '-' (each tag on separate line). Tagfile format is the same as config hostfile format.
Tag list to be added for specified host(s), comma separated values.

INVENTORY DEL-HOST

Delete host(s) from inventory database.

Host(s) to delete.
Delete all hosts.
Enable beta functionality.
Read additional hosts to delete from specified file or from stdin if '-' (each host on separate line). Hostfile format is the same as config hostfile format.
Use specified custom configuration file.
Use specified custom inventory directory. Inventory directory is set up by the following rules: if cdist configuration resolves this value then specified directory is used, if HOME env var is set then ~/.cdit/inventory is used, otherwise distribution inventory directory is used.

INVENTORY DEL-TAG

Delete tag(s) from inventory database.

List of host(s) for which tags are deleted.
Delete all tags for specified host(s).
Enable beta functionality.
Read additional hosts to delete tags for from specified file or from stdin if '-' (each host on separate line). Hostfile format is the same as config hostfile format.
Use specified custom configuration file.
Use specified custom inventory directory. Inventory directory is set up by the following rules: if cdist configuration resolves this value then specified directory is used, if HOME env var is set then ~/.cdit/inventory is used, otherwise distribution inventory directory is used.
Read additional tags from specified file or from stdin if '-' (each tag on separate line). Tagfile format is the same as config hostfile format.
Tag list to be deleted for specified host(s), comma separated values.

INVENTORY LIST

List inventory database.

Host(s) to list.
List hosts that have all specified tags, if -t/--tag is specified.
Enable beta functionality.
Read additional hosts to list from specified file or from stdin if '-' (each host on separate line). If no host or host file is specified then, by default, list all. Hostfile format is the same as config hostfile format.
Use specified custom configuration file.
Suppress tags listing.
Use specified custom inventory directory. Inventory directory is set up by the following rules: if cdist configuration resolves this value then specified directory is used, if HOME env var is set then ~/.cdit/inventory is used, otherwise distribution inventory directory is used.
Host is specified by tag, not hostname/address; list all hosts that contain any of specified tags.

PREOS

Create PreOS.

Add configuration directory (one that contains "preos" subdirectory).
Use specified custom configuration file.
List available PreOS-es.

Currently, the following PreOS-es are supported:

  • debian
  • ubuntu
  • devuan

PREOS DEBIAN/DEVUAN

target directory where PreOS will be bootstrapped
target debootstrap architecture, by default 'amd64'
do bootstrap step
Enable beta functionality.
do configure step
parameters that will be passed to cdist config, by default '-v' is used
create bootable PreOS on specified drive
remote exec that cdist config will use, by default internal script is used
init manifest that cdist config will use, by default internal init manifest is used
ssh key files that will be added to cdist config; '__ssh_authorized_keys root ...' type is appended to initial manifest
use specified mirror for debootstrap
Set specified password for root, generated by default
PXE boot directory
remove target directory after finishing
use specified script for debootstrap
suite used for debootstrap, by default 'stable'
remote copy that cdist config will use, by default internal script is used

PREOS UBUNTU

target directory where PreOS will be bootstrapped
target debootstrap architecture, by default 'amd64'
do bootstrap step
Enable beta functionality.
do configure step
parameters that will be passed to cdist config, by default '-v' is used
create bootable PreOS on specified drive
remote exec that cdist config will use, by default internal script is used
init manifest that cdist config will use, by default internal init manifest is used
ssh key files that will be added to cdist config; '__ssh_authorized_keys root ...' type is appended to initial manifest
use specified mirror for debootstrap
Set specified password for root, generated by default
PXE boot directory
remove target directory after finishing
use specified script for debootstrap
suite used for debootstrap, by default 'xenial'
remote copy that cdist config will use, by default internal script is used

SHELL

This command allows you to spawn a shell that enables access to the types as commands. It can be thought as an "interactive manifest" environment. See below for example usage. Its primary use is for debugging type parameters.

Select shell to use, defaults to current shell. Used shell should be POSIX compatible shell.

INFO

Display information for cdist (global explorers, types).

Glob pattern. If it contains special characters('?', '*', '[') then it is used as specified, otherwise it is translated to *pattern*.
Show help message and exit.
Display all info. This is the default.
Add configuration directory (can be repeated).
Display info for global explorers.
Interpret pattern as a fixed string.
Display full details.
Use specified custom configuration file.
Display info for types.

SCAN

Runs cdist as a daemon that discover/watch on hosts and reconfigure them periodically.

Interface to listen on. Can be specified multiple times.
Scanner components to enable. Can be specified multiple time to enable more than one component. Supported modes are: scan, trigger and config. Defaults to tiggger and scan.
Path to script used to resolve a remote host name from an IPv6 address.
List known hosts and exit.
How long (seconds) to wait before reconfiguring after last try (config mode only).
How long (seconds) to wait between ICMPv6 echo requests (trigger mode only).

CONFIGURATION

cdist obtains configuration data from the following sources in the following order (from higher to lower precedence):

1.
command-line options
2.
configuration file specified at command-line
3.
configuration file specified in CDIST_CONFIG_FILE environment variable
4.
environment variables
5.
user's configuration file (first one found of ~/.cdist.cfg, $XDG_CONFIG_HOME/cdist/cdist.cfg, in specified order)
6.
system-wide configuration file (/etc/cdist.cfg).



CONFIGURATION FILE FORMAT

cdist configuration file is in the INI file format. Currently it supports only [GLOBAL] section. The possible keywords and their meanings are as follows:

Use specified archiving. Valid values include: 'none', 'tar', 'tgz', 'tbz2' and 'txz'.
Enable beta functionality. It recognizes boolean values from 'yes'/'no', 'on'/'off', 'true'/'false' and '1'/'0'.
Specify cache path pattern.
Colorize cdist's output. cf. the --colors option.
List of configuration directories separated with the character conventionally used by the operating system to separate search path components (as in PATH), such as ':' for POSIX or ';' for Windows. If also specified at command line then values from command line are appended to this value.
Specify default initial manifest.
Specify inventory directory.
Specify number of jobs for parallel processing. If -1 then the default, number of CPU's in the system is used. If 0 then parallel processing in jobs is disabled. If set to positive number then specified maximum number of processes will be used.
Shell command used for local execution.
Directory to save cdist output in.
Process hosts in parallel. If -1 then the default, number of CPU's in the system is used. If 0 then parallel processing of hosts is disabled. If set to positive number then specified maximum number of processes will be used.
Command to use for remote copy (should behave like scp).
Command to use for remote execution (should behave like ssh).
Directory to save cdist output in on the target host.
Shell command at remote host used for remote execution.
Enable/disable saving output streams (enabled by default). It recognizes boolean values from 'yes'/'no', 'on'/'off', 'true'/'false' and '1'/'0'.
Timestamp log messages with the current local date and time in the format: YYYYMMDDHHMMSS.us.
Set verbosity level. Valid values are: 'ERROR', 'WARNING', 'INFO', 'VERBOSE', 'DEBUG', 'TRACE' and 'OFF'.

FILES

~/.cdist
Your personal cdist config directory. If exists it will be automatically used.
~/.cdist/cache
Local cache directory.
~/.cdist/inventory
The home inventory directory. If ~/.cdist exists it will be used as default inventory directory.
~/.cdist/preos
PreOS plugins directory, if existing.
The distribution configuration directory. It contains official types and explorers. This path is relative to cdist installation directory.
The distribution inventory directory. This path is relative to cdist installation directory.
The distribution PreOS plugins directory.
/etc/cdist.cfg
Global cdist configuration file, if exists.
~/.cdist.cfg or $XDG_CONFIG_HOME/cdist/cdist.cfg
Local cdist configuration file, if exists.

NOTES

cdist detects if host is specified by IPv6 address. If so then remote_copy command is executed with host address enclosed in square brackets (see scp(1)).

EXAMPLES

# Configure ikq05.ethz.ch with debug enabled
% cdist config -vvv ikq05.ethz.ch
# Configure hosts in parallel and use a different configuration directory
% cdist config -c ~/p/cdist-nutzung \


    -p ikq02.ethz.ch ikq03.ethz.ch ikq04.ethz.ch
# Use custom remote exec / copy commands
% cdist config --remote-exec /path/to/my/remote/exec \


    --remote-copy /path/to/my/remote/copy \


    -p ikq02.ethz.ch ikq03.ethz.ch ikq04.ethz.ch
# Configure hosts read from file loadbalancers
% cdist config -f loadbalancers
# Configure hosts read from file web.hosts using 16 parallel jobs
% cdist config -j 16 -f web.hosts
# Display banner
cdist banner
# Show help
% cdist --help
# Show Version
% cdist --version
# Enter a shell that has access to emulated types
% cdist shell
% __git
usage: __git --source SOURCE [--state STATE] [--branch BRANCH]


             [--group GROUP] [--owner OWNER] [--mode MODE] object_id
# Install ikq05.ethz.ch with debug enabled
% cdist install -vvv ikq05.ethz.ch
# List inventory content
% cdist inventory list -b
# List inventory for specified host localhost
% cdist inventory list -b localhost
# List inventory for specified tag loadbalancer
% cdist inventory list -b -t loadbalancer
# Add hosts to inventory
% cdist inventory add-host -b web1 web2 web3
# Delete hosts from file old-hosts from inventory
% cdist inventory del-host -b -f old-hosts
# Add tags to specified hosts
% cdist inventory add-tag -b -t europe,croatia,web,static web1 web2
# Add tag to all hosts in inventory
% cdist inventory add-tag -b -t vm
# Delete all tags from specified host
% cdist inventory del-tag -b -a localhost
# Delete tags read from stdin from hosts specified by file hosts
% cdist inventory del-tag -b -T - -f hosts
# Configure hosts from inventory with any of specified tags
% cdist config -b -t web dynamic
# Configure hosts from inventory with all specified tags
% cdist config -b -t -a web dynamic
# Configure all hosts from inventory db
$ cdist config -b -A
# Create default debian PreOS in debug mode
$ cdist preos debian /preos/preos-debian -vvvv -C \


    -k ~/.ssh/id_rsa.pub -p /preos/pxe-debian
# Create ubuntu PreOS
$ cdist preos ubuntu /preos/preos-ubuntu -C \


    -k ~/.ssh/id_rsa.pub -p /preos/pxe-ubuntu
# Create ubuntu PreOS on drive /dev/sdb
# and set root password to 'password'.
$ cdist preos ubuntu /mnt -B -C \


    -k ~/.ssh/id_rsa.pub -D /dev/sdb \


    -P password


ENVIRONMENT

Setup the base directory for the temporary directory. See http://docs.python.org/py3k/library/tempfile.html for more information. This is rather useful, if the standard directory used does not allow executables.
Colon delimited list of config directories.
Selects shell for local script execution, defaults to /bin/sh.
Selects shell for remote script execution, defaults to /bin/sh.
Allow overwriting type parameters.
Create dependencies based on the execution order. Note that in version 6.2.0 semantic of this processing mode is finally fixed and well defined.
Use this command for remote execution (should behave like ssh).
Use this command for remote copy (should behave like scp).
Use this directory as inventory directory.
Enable beta functionality.
Custom cache path pattern.
Colorize cdist's output. cf. the --colors option.
Custom configuration file.

EXIT STATUS

The following exit values shall be returned:

0 Successful completion.

1 One or more host configurations failed.

AUTHORS

Originally written by Nico Schottelius <nico-cdist--@--schottelius.org> and Steven Armstrong <steven-cdist--@--armstrong.cc>.

CAVEATS

When operating in parallel, either by operating in parallel for each host (-p/--parallel) or by parallel jobs within a host (-j/--jobs), and depending on target SSH server and its configuration you may encounter connection drops. This is controlled with sshd MaxStartups configuration options. You may also encounter session open refusal. This happens with ssh multiplexing when you reach maximum number of open sessions permitted per network connection. In this case ssh will disable multiplexing. This limit is controlled with sshd MaxSessions configuration options. For more details refer to sshd_config(5).

When requirements for the same object are defined in different manifests (see example below), for example, in init manifest and in some other type manifest and those requirements differ then dependency resolver cannot detect dependencies correctly. This happens because cdist cannot prepare all objects first and run all objects afterwards. Some object can depend on the result of type explorer(s) and explorers are executed during object run. cdist will detect such case and display a warning message. An example of such a case:

init manifest:


    __a a


    require="__e/e" __b b


    require="__f/f" __c c


    __e e


    __f f


    require="__c/c" __d d


    __g g


    __h h
type __g manifest:


    require="__c/c __d/d" __a a
Warning message:


    WARNING: cdisttesthost: Object __a/a already exists with requirements:


    /usr/home/darko/ungleich/cdist/cdist/test/config/fixtures/manifest/init-deps-resolver /tmp/tmp.cdist.test.ozagkg54/local/759547ff4356de6e3d9e08522b0d0807/data/conf/type/__g/manifest: set()


    /tmp/tmp.cdist.test.ozagkg54/local/759547ff4356de6e3d9e08522b0d0807/data/conf/type/__g/manifest: {'__c/c', '__d/d'}


    Dependency resolver could not handle dependencies as expected.


COPYING

Copyright (C) 2011-2020 Nico Schottelius. Free use of this software is granted under the terms of the GNU General Public License v3 or later (GPLv3+).

COPYRIGHT

ungleich GmbH 2021

September 11, 2024 7.0.0