NAME¶rozofs.conf - RozoFS common configuration file
DESCRIPTION¶rozofs.conf is managed thanks to libconfig <http://www.hyperrealm.com/libconfig/> and thus complies to the libconfig grammar.
Boolean values must be set to either True or False.
String values must enclosed into double quotes.
numa_aware¶That boolean enables to take into account the NUMA architecture of the mother board in order to collocate some RozoFS modules on a same node. This may improve the inter process communication performance.
file_distribution_rule¶This parameter enables to choose the file distribution rule at file creation. The following rules are defined:
trashed_file_per_run¶Maximum number of files that the remove bins thread of a slave export can remove in a run. A 2 second delay occurs between every run. This provides the maximum deletion rate.
deletion_delay¶Minimum delay in seconds between the FID unlink in the export and the actual projections removal on the storage nodes.
nb_trash_thread¶Number of trash threads that work in parallel (default: 2, maximum: 8)
statfs_period¶caching delay in second of the data reported by a statfs(). The default value is 10 seconds. When the delay is 0, there is no caching.
core_file_directory¶The directory under which the RozoFS modules store their core files in case a crash occurs.
nb_core_file¶Maximum number of RozoFS core file to keep on this node whatever the RozoFS executable. The older core files are kept while the newest ones are removed.
nb_disk_thread¶Specifies the number of threads within the STORIO process that can operate the disk read/write.
crc32c_check¶Boolean (True or False) indicating if a check (thanks to CRC error detecting code) will be used for detecting accidental changes to raw data.
crc32c_generate¶Boolean (True or False) indicating if a checksum must be generated (thanks to CRC error detecting code) for each write request.
crc32c_hw_forced¶That boolean (True or False) can be asserted if the information reported by cpuid does not indicate that hardware CRC is supported whilst the current CPU supports it. This might be the case with VirtualBox.
disk_usage_threshold¶when set to non zero value, the storios log a warning when a device percent usage is over or equal to this limit.
disk_read_threshold¶when set to non zero value, the storios log a warning when the average read delay on a device is over or equal to this limit in micro seconds.
disk_write_threshold¶when set to non zero value, the storios log a warning when the average write delay on a device is over or equal to this limit in micro seconds.
fid_recycle¶Boolean (True or False). When set to TRUE, RozoFS starts recycling the FID upon file deletion. The threshold for which the FID recycling starts can be also configured within the rozofs.conf file .
trash_high_threshold¶When fid_recycle is set to TRUE, that parameter provides the threshold for which the FID recycling starts .
wr_ack_on_inverse¶Boolean (True or False). When set to TRUE, RozoFS acknowledges a write request once a count of inverse projections have been successfully written. Otherwise, by default it waits until a count of forward projections are written. By default, the write acknowlegdment anticipation is not set.
allow_disk_spin_down¶This boolean has to be set to enable the spinning down of the disks of a storage node. The STORIO monitoring thread periodically checks the file systems mounted on its disks. This boolean prevents the check to be performed in case no modification takes place on the disk (i.e neither write, truncate nor delete). So when no access is done to the disks, the monitoring thread does not either access to the disk, and the disk spinning down can take place.
It is no use to set this boolean when you do not configure a spin down timeout (see hdparm -S).
export_dscp¶Value that indicates the DSCP that is associated with TCP connections used to communicate with the Metadata server. By default the value corresponds to Expedited Forwarding (EF) class.
storio_dscp¶Value that indicates the DSCP that is associated with TCP connections used to communicate with the Storage server. By default the value corresponds to Assured Forwarding (AF41) class.
export_attr_thread¶Boolean (True or False). When set, this flag indicates that the export has its attribute writeback threads activated (default true).
rozofsmount_fuse_reply_thread¶Boolean (True or False). When set, this flag indicates that the rozofsmount has its fuse reply threads activated (default true).
client_xattr_cache¶Boolean (True or False). When set, this flag indicates that the rozofsmount caches the extended attributes. The timer used is the same as the one used for attributes (default false).
async_setattr¶Boolean (True or False). When set, this flag indicates that the rozofsmount operates in asynchronous mode for setattr operations (default false).
export_versioning¶Boolean (True or False). When set, this flag indicates that any deleted object (file/directory) are save under @rozofs-del@ pseudo directory of the parent directory (default false).
device_automount¶Boolean (True or False). Default value is False. When set, the RozoFS devices are automatically mounted on the right directories provided that they have the destination mark file on them; the destination mark file is a file named storage_c<cluster>_s<storage>_<device> that tells what this RozoFS device is intended for. No mount instruction should be set in the fstab file for these devices.
when device_automount is True, the following configuration parameters become effective.
device_automount_path¶This parameter gives the path where the devices have to be mounted. The default value is "/srv/rozofs/storages". That is device <device> of storage <storage> of cluster <cluster> will be mounted on directory <device_automount_path>/storage_c<cluster>_s<storage>/<device>. The root parameter in the storage.conf file is ignored and can even be omitted. All the paths are automatically created when needed.
device_automount_option¶This parameter provides an optional mount option that should be supported by the lowlevel file system.
mkdir_ok_instead_of_eexist¶When set to True this parameter makes the export respond OK instead of EEXIST on a directory creation while the directory already exist. This enables concurrent processes to all execute mkdir at the same time without getting any error. Be aware that this will prevent some application to make use of mkdir to simulate a lock between nodes.
mknod_ok_instead_of_eexist¶When set to True this parameter makes the export respond OK instead of EEXIST on a file creation while the file already exist. This enables concurrent processes to all execute mkdir at the same time without getting any error. Be aware that this will prevent some application to make use of mknod to simulate a lock between nodes..
client_fast_reconnect¶When set to True, the rozofsmount updates some of its timers in order to reconnect to the exportd as fast as possible.
disable_sync_attributes¶When set to True it disables synchonous write of the attributes.
device_selfhealing_mode¶This parameter is a string that can take the following values:
For self-healing to be active, the export_hosts parameter must be configured too.
when self-healing is set, the following configuration parameters become effective.
device_selfhealing_delay¶Gives the delay in minutes that a failure must last before starting an automatic rebuild of a failed device. The lowest possible value is 1.
device_self_healing_process¶This parameter sets the number of rebuilding processes that will be launched in parallel in case of an automatic rebuild for self-healing purpose. This is also the default number of rebuilding processes for operator rebuild commands.
export_hosts¶This parameter is needed for automatic rebuilds to take place. It gives the list of addresses (or dns names) where exportd daemon is running. When securing the exportd thanks to virtual IP address, only the virtual IP address is declared (i.e "192.168.10.1"). When no virtual IP address is used, the list of "physical" IP addresses of each host securing the exportd must be given (i.e "192.168.10.1/192.168.20.1").
device_selfhealing_read_throughput¶This parameter provides the maximum bandwidth in MB/s that a single rebuild process can take.
spare_restore_enable¶Set to False to disable spare file restoring. Default value is True, which enables the spare file restoring feature. This feature tries to relocate data saved in spare files to the nominal location. This feature also requires export_hosts parameter to be set. when spare file restoring is active, the following configuration parameters become effective.
spare_restore_loop_delay¶This parameter gives in minutes the delay between two spare file restoring attempts.
spare_restore_read_throughput¶This parameter is the maximum network/disk bandwidth that the spare file restoring process will take to run.
min_metadata_inodes¶Minimum number of inodes that must be available on metadata device to allow a mknod/mkdir operation (default 256K)
min_metadata_MB¶Minimum available size in MB that must be available on metadata device to allow a mknod/mkdir operation (default 2048 MB)
export nb_thin_thread¶Number of thin provisioning threads that work in parallel. (default:2; maximum:4)
thin_scan_file_per_run¶Number of files that the exportd can submit to storages to get the number of blocks (thin provisioning) in a run. A new run occurs every 2 seconds. This requires that the export has been configured with thin provisioning. (default:1000)
thin_lv1_hash_tb_size¶Thin provisioning Level 1 hash table size in power of 2 (changing this parameter will take effect on the next export restart)
expthin_guard_delay_sec¶Guard delay en seconds between two file scanning of a file that belongs to an export configured for thin provisioning. (default:10; maximum 600)
level2_cache_max_entries_kb¶Maximum number of i-node entries that the export level 2 cache can contain (unit is K). (default 512K)
# sample RozoFS common configuration file # # Colocated when possible # numa_aware = True; # # Round robin gices the better performance # file_distribution_rule = 2; # # Delete a maximum of 32 files every 2 seconds # trashed_file_per_run = 32; # # Do not keep more than 1 core file module # nb_core_file = 1; # # Need more disk threads # nb_disk_threads = 8; # # CRC32 enabling # crc32c_check = True; crc32c_generate = True; crc32c_hw_forced = True; device_automount = True; # # Very slow self healing : 2 processes to read 15MB/s each # device_selfhealing_mode = "spareOnly"; device_selfhealing_delay = 10; device_self_healing_process = 2; device_selfhealing_read_throughput = 15; export_hosts = "192.168.10.1/192.168.10.2";