NAME¶

dat.conf - configuration file for static registration of user-level DAT rdma providers

DESCRIPTION¶

The DAT (direct access transport) architecture supports the use of multiple DAT providers within a single consumer application. Consumers implicitly select a provider using the Interface Adapter name parameter passed to dat_ia_open().

The subsystem that maps Interface Adapter names to provider implementations is known as the DAT registry. When a consumer calls dat_ia_open(), the appropriate provider is found and notified of the consumer's request to access the IA. After this point, all DAT API calls acting on DAT objects are automatically directed to the appropriate provider entry points.

A persistent, administratively configurable database is used to store mappings from IA names to provider information. This provider information includes: the file system path to the provider library object, version information, and thread safety information. The location and format of the registry is platform dependent. This database is known as the Static Registry (SR) and is provided via entries in the dat.conf file. The process of adding a provider entry is termed Static Registration.

Registry File Format¶

* All characters after # on a line are ignored (comments).
* Lines on which there are no characters other than whitespace
and comments are considered blank lines and are ignored.
* Non-blank lines must have seven whitespace separated fields.
These fields may contain whitespace if the field is quoted
with double quotes. Within fields quoted with double quotes,
the backslash or quote are valid escape sequences:
* Each non-blank line will contain the following fields:
- The IA Name.
- The API version of the library:
[k|u]major.minor where "major" and "minor" are both integers
in decimal format. User-level examples: "u2.0".
- Whether the library is thread-safe: [threadsafe|nonthreadsafe]
- Whether this is the default section: [default|nondefault]
- The library image, version included, to be loaded.
- The vendor id and version of DAPL provider: id.major.minor
- ia params, IA specific parameters - device name and port
- platform params, (not used)

OpenFabrics RDMA providers:¶

Provider options for different CM services

1. cma - OpenFabrics rdma_cm - uses rdma_cm services for connections
- requires IPoIB and SA/SM services for IB
- netdev used for device name, without port designation (ia_params)
- Supports any transport rdma_cm supports including IB, iWARP, RoCEE
- libdaplofa (2.0)

2. scm - uDAPL socket based CM - exchanges CM information over sockets
- eliminates the need for rdma_cm, IPoIB, and SA for IB
- verbs device used for device name with port designation (ia_param)
- Supports IB, RoCEE. Doesn't support iWARP
- libdaploscm (2.0)

3. ucm - uDAPL unreliable IB CM - exchanges CM information via IB UD QP's
- eliminates the need for sockets or rdma_cm
- verbs device used for device name with port designation (ia_param)
- Supports IB only, no name service.
- libdaploucm (2.0)

Example entries for each OpenFabrics provider¶

1. cma - OpenFarbrics rdma_cm examples

ofa-v2-ib0 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "ib0 0" ""
ofa-v2-iwarp u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "eth2 0" ""
ofa-v2-cma-roe-eth2 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "eth2 0" ""
ofa-v2-cma-roe-eth3 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "eth3 0" ""

NOTE: The OpenFabrics CMA providers use <ia_params> to specify the device with one of the following:
network address, network hostname, or netdev name; along with port number.

2. scm - uDAPL socket based CM v2.0 examples

ofa-v2-mlx4_0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 "mlx4_0 1" ""
ofa-v2-mlx4_0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 "mlx4_0 2" ""
ofa-v2-mlx5_1-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 "mlx5_1 1" ""
ofa-v2-ehca0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 "ehca0 1" ""
ofa-v2-scm-roe-mlx5_0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 "mlx5_0 1" ""

3. ucm - uDAPL unreliable IB CM examples

ofa-v2-mlx4_0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 "mlx4_0 1" ""
ofa-v2-mlx4_0-2u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 "mlx4_0 2" ""
ofa-v2-ipath0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 "ipath0 1" ""
ofa-v2-ehca0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 "ehca0 1" ""

Note: OpenIB- and ofa-v2- IA names are unique mappings, reserved for OpenFabrics providers.

The default location for this configuration file is /etc/dat.conf.

The file location may be overridden with the environment variable DAT_OVERRIDE=/your_own_directory/your_dat.conf.

SEE ALSO rdma_cm verbs socket¶