SECTIONS¶
volume <volume-directory>|<url> (optional)
Absolute path pointing to a btrfs file system containing
the source subvolume(s) to be backed up. Usually the mount point of a btrfs
filesystem mounted with the subvolid=5 option.
subvolume <subvolume-name>
Subvolume to be backed up, relative to the
<volume-directory> of the
volume section, or absolute if
the
volume section is omitted. Accepts wildcard character
"*".
Note that if this subvolume is btrfs root (id=5), it needs to have
a valid UUID, which is not the case for file systems created with
btrfs-progs < 4.16.
target [send-receive|raw]
<target-directory>|<url>
Target directory where the backup subvolumes are to be
created. The optional target type defaults to “send-receive”,
see TARGET TYPES below for details.
Multiple target sections are allowed, in any context: a
target defined in volume or global context will be used for
all underlying subvolume sections (hint: run "btrbk list"
or "btrbk config print" to see the resulting configuration).
If a <url> is specified, btrbk actions (shell
commands) are executed remotely via ssh, using the SSH Options described
below. Accepted formats are:
ssh://<hostname>[:<port>]/<directory>
<hostname>:<directory>
Where <hostname> is either a host name, an IPv4
address in dotted-decimal form, or an IP literal encapsulated within square
brackets (e.g. "[2001:db8::7]").
If you are connecting to virtual machines, consider configuring
several volume sections for a <hostname>, with distinct
<port> numbers for each machine.
OPTIONS¶
The options described here can be specified in global
context as well as volume, subvolume and target
sections, unless stated otherwise.
Basic Options¶
timestamp_format short|long|long-iso
Timestamp format used as postfix for new snapshot
subvolume names. Defaults to “long”.
short
YYYYMMDD[_N] (e.g.
"20150825", "20150825_1")
long
YYYYMMDD<T>hhmm[_N] (e.g.
"20150825T1531")
long-iso
YYYYMMDD<T>hhmmss±hhmm[_N]
(e.g. "20150825T153123+0200")
Note that a postfix "_N" is appended to the timestamp if
a snapshot or backup already exists with the timestamp of current
date/time.
Use “long-iso” if you want to make sure that btrbk
never creates ambiguous time stamps (which can happen if multiple snapshots
are created during a daylight saving time clock change).
Note that using “long-iso” has implications on the
scheduling, see Reference Time below.
snapshot_dir <directory>
Directory in which the btrfs snapshots are created,
relative to <volume-directory> of the volume section, or
absolute if the volume section is omitted. Note that btrbk does not
automatically create this directory, and the snapshot creation will fail if it
is not present.
snapshot_name <basename>
Base name of the created snapshot (and backup). This
option is only valid in the subvolume section. Defaults to
<subvolume-name>.
snapshot_create always|onchange|ondemand|no
If set to “always”, snapshots are always
created. If set to “onchange”, snapshots are only created if the
last snapshot is not up-to-date, i.e. the source subvolume has changed (more
precisely: the btrfs generation has been increased) since the last snapshot
was created. If set to “ondemand”, snapshots are only created if
at least one target subvolume is reachable (useful if you are tight on disk
space and you only need btrbk for backups to an external disk which is not
always connected). If set to “no”, the snapshots are never
created (useful if another instance of btrbk is taking care of snapshot
creation). Defaults to “always”.
incremental yes|no|strict
If set, incremental backups are created. If set to
“strict”, non-incremental (initial) backups are never created,
and incremental backups are restricted to
related parents (by
parent-uuid relationship). Defaults to “yes”.
Note that even if the parent-uuid chain is broken, snapshots and
backups can still share data (which is especially true for backups created
with incremental option enabled), and are perfectly suitable as
parents for incremental send-receive operations. But as btrbk can not be
certain about this, such operations are disallowed in "incremental
strict" mode.
noauto yes|no
If set, the context is skipped by all btrbk actions
unless explicitly enabled by a matching btrbk <filter> command
line argument (e.g. "btrbk run myfilter").
Grouping Options¶
group <group-name> [<group-name>]...
Add the current section (volume, subvolume or target) to
user-defined groups, which can be used as filter for most btrbk commands (see
btrbk(1) FILTER STATEMENTS). This option can be set multiple times within the
same context.
Retention Policy Options¶
preserve_day_of_week monday|tuesday|...|sunday
Defines on what day a snapshot/backup is considered to be
a "weekly" backup. Weekly, monthly and yearly backups are preserved
on this day of week (see RETENTION POLICY below). Defaults to
“sunday”.
preserve_hour_of_day [0..23]
Defines after what time (in full hours since midnight) a
snapshot/backup is considered to be a "daily" backup. Daily, weekly,
monthly and yearly backups are preserved on this hour (see RETENTION POLICY
below). Ignored on snapshots or backups without time information
(timestamp_format short). Defaults to “0”.
snapshot_preserve no|<retention_policy>
Set retention policy for snapshots (see RETENTION POLICY
below). If set to “no”, preserve snapshots according to
snapshot_preserve_min only. Defaults to “no”.
Note that snapshot_preserve has no effect if
snapshot_preserve_min is set to “all” (the
default).
snapshot_preserve_min
all|latest|<number>{h,d,w,m,y}
Preserve all snapshots for a minimum amount of hours (h),
days (d), weeks (w), months (m) or years (y), regardless of how many there
are. If set to “all”, preserve all snapshots forever. If set to
“latest”, preserve latest snapshot. Defaults to
“all”.
target_preserve no|<retention_policy>
Set retention policy for backups (see RETENTION POLICY
below). If set to “no”, preserve backups according to
target_preserve_min only. Defaults to “no”.
Note that target_preserve has no effect if
target_preserve_min is set to “all” (the default).
target_preserve_min
all|latest|no|<number>{h,d,w,m,y}
Preserve all backups for a minimum amount of hours (h),
days (d), weeks (w), months (m) or years (y), regardless of how many there
are. If set to “all”, preserve all backups forever. If set to
“latest”, always preserve the latest backup (useful in
conjunction with "target_preserve no", if you want to keep the
latest backup only). If set to “no”, only the backups following
the target_preserve policy are created. Defaults to
“all”.
archive_preserve no|<retention_policy>
Set retention policy for archives ("btrbk
archive" command), with same semantics as target_preserve.
archive_preserve_min
all|latest|no|<number>{h,d,w,m,y}
Set retention policy for archives ("btrbk
archive" command), with same semantics as
target_preserve_min.
archive_exclude <pattern>
Exclude subvolumes matching <pattern> from
archiving. The pattern accepts wildcard character "*", and is
matched against the end of the pathname.
SSH Options¶
ssh_identity <file>|no
Absolute path to a ssh identity file (private key). If
not set, the ssh default is used (see
ssh(1), "-i identity_file").
Note that if the identity key is password protected and no authentication
agent is used, btrbk will prompt for user input on every connection
attempt.
ssh_user <username>|no
Remote username for ssh. Defaults to
“root”. Make sure the remote user is able to run
"btrfs" with root privileges (see option backend for
details). If set to “no”, the ssh default is used.
ssh_compression yes|no
Enables or disables the compression of ssh connections.
Defaults to “no”. Note that if stream_compress is
enabled, ssh compression will always be disabled for send/receive
operations.
ssh_cipher_spec default|<cipher_spec>
Selects the cipher specification for encrypting the
session (comma-separated list of ciphers in order of preference). See the
"-c cipher_spec" option in
ssh(1) for more information. Defaults to
“default” (the ciphers specified in
ssh_config(5)).
Data Stream Options¶
stream_compress <compress_command>|no
Compress the btrfs send stream before transferring it
from/to remote locations. Defaults to “no”. If enabled, make
sure that <compress_command> is available on the source and
target hosts. Supported <compress_command>: gzip, pigz, bzip2,
pbzip2, xz, lzo, lz4, zstd.
stream_compress_level default|<number>
Compression level for the specified
<compress_command>. Refer to the related man-page for details
(usually [1..9], where 1 means fastest compression). Defaults to
“default” (the default compression level of
<compress_command>).
stream_compress_long default|<number>
Enable long distance matching for the specified
<compress_command>. Refer to the related man-page for details.
Only supported for "zstd".
stream_compress_threads default|<number>
Number of threads to use for <compress_command>.
Only supported for "pigz", "pbzip2", "zstd" and
recent versions of "xz".
stream_compress_adapt default|<number>
Enable adaptive compression for <compress_command>.
Only supported for "zstd" (version >= 1.3.6).
stream_buffer <size>|no
Add a buffer to the btrfs send stream (locally, on
uncompressed data), with a maximum size of
<size>. This can give
a speed improvement (measured up to 20%) on both local or remote operations,
but also increases system load. A suffix of "k", "m",
"g", or "%" can be added to
<size> to denote
kilobytes (*1024), megabytes, gigabytes, or a percentage of total physical
memory. Defaults to “no”.
If enabled, make sure that the "mbuffer" command (at
least version 20180505) is available on the host running btrbk. As of
btrbk-0.29.0, mbuffer(1) is used for both rate_limit and
stream_buffer options:
mbuffer [-m <stream_buffer>] [-r <rate_limit>]
Note that mbuffer(1) always reads defaults from
"/etc/mbuffer.rc" and
"~/.mbuffer.rc".
Leave this option disabled if your main concern is a stable backup
process: while recent versions of mbuffer have proven reliable, it is often
desirable to keep things simple rather than adding an additional,
multi-threaded process to the command pipe.
stream_buffer_remote <size>|no
Add a buffer on remote hosts (either source or target).
Defaults to “no”.
Enable this if you prefer buffering on the remote side, or even on
both sides: reasons for this depend on available memory, disk and cpu
performance (btrfs send/receive, compression), as well as networking
constraints.
rate_limit <rate>|no
Limit the read rate of the btrfs send stream to
<rate> bytes per second (locally, on uncompressed send stream). A
suffix of "k", "m", "g", or "t" can be
added to denote kilobytes (*1024), megabytes, and so on. Defaults to
“no”. Note that rate_limit implicitly adds a stream
buffer (see stream_buffer option above).
rate_limit_remote <rate>|no
Add rate limit on remote hosts (either source or target).
Defaults to “no”. Note that it usually does not make much sense
to enable both rate_limit and rate_limit_remote.
System Options¶
transaction_log <file>|no
If set, all transactions (snapshot create, subvolume
send-receive, subvolume delete) as well as abort messages are logged to
<file>, in a space-separated table format: "localtime type status
target_url source_url parent_url message".
transaction_syslog <facility>|no
If set, all transactions (as described in
transaction_log above) are logged to syslog. The program name used in
the messages is "btrbk". Accepted parameters for
<facility>: user, mail, daemon, auth, lpr, news, cron, authpriv,
local0..local7.
lockfile <file>|no
Create lockfile <file> on startup; checks lockfile
before running any btrfs commands (using perl "flock"), and exits if
the lock is held by another btrbk instance. Ignored on dryrun (-n,
--dry-run). See also --lockfile command-line option.
backend <backend>
Backend filesystem utilities to be used for btrfs
specific operations. Available backends:
btrfs-progs
Default backend, btrfs commands are called as specified
in
btrfs(8) (e.g. "btrfs subvolume show").
btrfs-progs-btrbk
btrfs commands are separated by a dash instead of a
whitespace (e.g. "btrfs-subvolume-show" instead of "btrfs
subvolume show"). Useful for setting suid or file capabilities (setcap)
on specific btrfs commands, as implemented in
<
https://github.com/digint/btrfs-progs-btrbk>.
btrfs-progs-sudo
btrfs commands are prefixed with "sudo -n"
(e.g. "sudo -n btrfs subvolume show" instead of "btrfs
subvolume show"). Make sure to have appropriate (root) permissions for
the "btrfs" command groups as well as the "readlink" and
"test" commands in /etc/sudoers.
btrfs-progs-doas
Similar to btrfs-progs-sudo, using prefix "doas
-n".
If you want to set this option for local or remote hosts only, set
backend_local or backend_remote (e.g. "backend_remote
btrfs-progs-btrbk").
If you want to set this option for regular (non-root) user only,
set backend_local_user.
compat <compat-option>...
Enable compatibility options. Available
compat-option:
busybox
Use busybox compatible commands, at the expense of slight
overhead while reading filesystem information.
ignore_receive_errors *experimental*
Tell
btrfs-receive(8) to not terminate on errors by
setting "--max-errors=0" option. Print warnings instead.
A known use case for this are target hosts lacking xattr support
(e.g. some Synology NAS), while the send-stream contains
"lsetxattr" commands. Another case is targets failing to set
otime, complaining with "ERROR: attribute 12 requested but not
present".
Note that there is no guarantee that backups created with
this option enabled can be restored at all.
If you want to set this option for local or remote hosts only, set
compat_local or compat_remote (e.g. "compat_remote
busybox").
cache_dir <directory>
If set, cache extent maps for the "btrbk
extents" command.
Btrfs Specific Options¶
incremental_prefs <list-spec>[:<amount>]...
Specify the preferences to determine the best common
(correlated) parent and clone sources for incremental backups, by choosing
from predefined candidate lists.
The list-spec defines from what candidate list the next
parent/clone-src should be appended to a result list; amount defines
how many (e.g. "sro:1 sro:1" is identical to "sro:2"),
or all if omitted. Any candidate which is already in the results is
dropped.
The resulting list of subvolumes is then used as parameters for
the btrfs-send(8) command: the first for "-p <parent>", all
others for "-c <clone-src>".
Available list-spec (candidate lists = filtered subsets of
correlated subvolumes):
sro,srn
All from snapshot_dir matching
snapshot_name, with parent_uuid relationship, sorted by btrbk timestamp
(o=older n=newer).
sao,san
All from snapshot_dir matching
snapshot_name, sorted by btrbk timestamp (o=older n=newer).
aro,arn
All from incremental_resolve, with parent_uuid
relationship, sorted by cgen (o=older n=newer).
Defaults to "sro:1 srn:1 sao:1 san:1 aro:1 arn:1". Note
that for most operations the default resolves a single parent, as there
usually are no newer snapshots, and all "sro:1 sao:1 aro:1"
resolve to the same snapshot.
Example: "defaults,sao,san,aro,arn" takes the defaults,
and adds clone sources for all (!) known candidates on the filesystem.
incremental_clones yes|no
If enabled, btrbk adds "-c <clone-src>"
to the
btrfs-send(8) command for all correlated subvolumes resolved by
incremental_prefs. If disabled, only "-p <parent>" is
used. Defaults to “yes”.
incremental_resolve mountpoint|directory
Specifies where to search for the best common parent for
incremental backups. If set to “mountpoint”, use parents in the
filesystem tree below the mount point of the snapshot and target directory. If
set to “directory”, use parents strictly below snapshot/target
directories. Set this to “directory” if you get access problems
(when not running btrbk as root). Defaults to
“mountpoint”.
btrfs_commit_delete yes|no
If set, wait for the transaction commit at the end of
each snapshot or backup deletion (sets --commit-each option for
"btrfs subvolume delete"). Defaults to “no”.
snapshot_qgroup_destroy yes|no *experimental*
target_qgroup_destroy yes|no *experimental*
archive_qgroup_destroy yes|no *experimental*
warn_unknown_targets yes|no
If set, prints a warning if btrbk encounters a target
subvolume at a unknown location (i.e. not following btrbk naming scheme, or
outside the target directory). Defaults to “no”.
RETENTION POLICY¶
Retention policies are defined individually for snapshots, backups
and archives (summarized as "backups" in the following text),
using a combination of:
*_preserve_min all|latest|no|<number>{h,d,w,m,y}
Amount of time (duration) in which all backups are
preserved.
*_preserve no|<retention_policy>
Schedule (single points in time) for which individual
backups are preserved.
Note that if "preserve_min" is set to
“all” (the default), any setting of "preserve"
obviously has no effect.
The format for <retention_policy> is:
[<hourly>h] [<daily>d] [<weekly>w] [<monthly>m] [<yearly>y]
hourly
Defines how many hours back hourly backups should be
preserved. The first backup of an hour is considered an hourly backup.
daily
Defines how many days back daily backups should be
preserved. The first backup of a day (starting at preserve_hour_of_day)
is considered a daily backup.
weekly
Defines how many weeks back weekly backups should be
preserved. The first daily backup created at preserve_day_of_week (or
the first backup in this week if none was made on the exact day) is considered
as a weekly backup.
monthly
Defines how many months back monthly backups should be
preserved. Every first weekly backup in a month is considered a monthly
backup.
yearly
Defines for how many years back yearly backups should be
preserved. Every first monthly backup in a year is considered a yearly
backup.
Use an asterisk for “all” (e.g.
"target_preserve 60d *m" states: "preserve daily backups for
60 days back, and all monthly backups").
Hint: Run btrbk with the -S, --print-schedule option
to get a comprehensive output of the scheduler results.
Reference Time¶
The local time on the host running btrbk defines the reference
time for all date/time calculations, especially for "beginning of a
day", and as a consequence for the first daily, weekly, monthly or
yearly backups. The local time on remote hosts (ssh source/target) is never
used.
Unless "timestamp_format long-iso" is set, daily backups
are preserved at "preserve_hour_of_day" (defaults to midnight) of
the respective time zone (and not for "00:00 UTC", which would be
"14:00" in Honolulu). This becomes relevant for setups with
multiple btrbk instances, e.g. many snapshot-only instances (spread around
the world), and a fetch-only instance on the backup server.
Caveat:
•If "timestamp_format long-iso" is set,
each btrbk instance on has a different interpretation of "first in
day". Make sure to run btrbk with the same time zone on every host, e.g.
by setting the TZ environment variable (see
tzset(3)).
TARGET TYPES¶
send-receive
Backup to a btrfs filesystem, using "btrfs
send/receive". This is the recommended (standard) target type. The
<target-directory> must be an absolute path and point to a
subvolume or directory within a btrfs file system. See
btrfs-send(8),
btrfs-receive(8).
raw *experimental*
Backup to a raw (filesystem independent) file from the
output of
btrfs-send(8), with optional compression and encryption.
Note that the target preserve mechanism is currently disabled for
incremental raw backups (btrbk does not delete any incremental raw
files)!
Raw backups consist of two files: the main data file containing
the btrfs send stream, and a sidecar file ".info" containing
metadata:
<snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg]
<snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg].info
For incremental backups ("incremental yes"),
please note that:
•As soon as a single incremental backup
file is lost or corrupted, all later incremental backups become invalid, as
there is no common parent for the subsequent incremental images anymore. This
might be a good compromise for a vacation backup plan, but for the long term
make sure that a non-incremental backup is triggered from time to time.
•There is currently no support for rotation of
incremental backups: if incremental is set, a full backup must be
triggered manually from time to time in order to be able to delete old
backups.
Additional options for raw targets:
raw_target_compress <compress_command>|no
Compression algorithm to use for raw backup target.
Supported <compress_command>: gzip, pigz, bzip2, pbzip2, xz, lzo,
lz4, zstd.
raw_target_compress_level default|<number>
Compression level for the specified
<compress_command>.
raw_target_compress_long default|<number>
Enable long distance matching for the specified
<compress_command>.
raw_target_compress_threads default|<number>
Number of threads to use for
<compress_command>.
raw_target_split <size>|no
Split the raw backup file into pieces of size
<size>.
raw_target_block_size <number>
Block size to use for writing the raw backup file.
Defaults to “128K”.
raw_target_encrypt gpg|openssl_enc|no
If enabled, encrypt the target raw file using gpg or
openssl_enc.
Additional options for "raw_target_encrypt gpg":
gpg_keyring <file>
Keyring to use for gpg, e.g.
"/etc/btrbk/gpg/pubring.kbx".
gpg_recipient <name>...
Encrypt for user id <name> (email
address).
Additional options for "raw_target_encrypt openssl_enc"
(very experimental):
openssl_ciphername <name>
Defaults to “aes-256-cbc”.
openssl_iv_size <size-in-bytes>|no
Depends on selected cipher.
openssl_keyfile <file>|no
Point to a key file in hex (absolute path). Example key
file creation (256bit key):
# dd if=/dev/urandom bs=1 count=32 \
| od -x -A n \
| tr -d "[:space:]" > /path/to/keyfile
kdf_backend <file>|no
KDF backend to be executed, e.g.
"/usr/share/btrbk/scripts/kdf_pbkdf2.py".
kdf_keysize <size-in-bytes>
Defaults to “32”.
kdf_keygen once|each
Defaults to “once”.
