NAME¶

oarsh - remote shell connector for OAR batch scheduler.

oarcp - oarsh companion to copy files from a node or to a node.

SYNOPSIS¶

oarsh [OPTIONS] <NODENAME> [COMMAND]

oarcp [OPTIONS] [NODENAME:]<PATHNAME> [NODENAME:]<PATHNAME>

DESCRIPTION¶

Connect to a node from the job submission frontend of the cluster or any other node.

OPTIONS¶

The oarsh mechanism uses OpenSSH client (the ssh command) under the hood to perform connections to remote hosts. Thus most of OpenSSH options can be used (but some are filtered out for security reasons).

ENVIRONMENT¶

OAR_JOB_ID

If not using the job-key mechanism, the OAR_JOB_ID environment variable has to be set to give the jobid of the job to connect to on the remote node (a node can run several jobs at a same time). This is only required for connecting from outside the job (e.g. from the job submission frontend).

OAR_JOB_KEY_FILE

If using the job-key mechanism (see oarsub -k -e/-i), oarsh can use the job-key defined by the OAR_JOB_KEY_FILE environment variable to connect to job on the remote machine. That mechanism can allow one to connect to a job from a machine which is outside the OAR cluster the job belong to (e.g. in a grid of OAR clusters), given the job-key is available on that machine. NB: it is also possible to give the job-key using oarsh -i option.

OAR_USER_CPUSET

Setting the OAR_USER_CPUSET environment variable for oarsh allows running the process in the target node with a subset of the CPU set of the job. Expected format for OAR_USER_CPUSET is a list of , or + or space separated processing unit. The processing unit ids of the job can be shown using oarprint (cpuset property), or more generally with hwloc's lstopo command (physical PU). The oarnodes command also shows the cpuset resource property for every OAR resource.

OAR_USER_GPUDEVICE

If the GPU device mechanism is enabled in the OAR installation, setting the OAR_USER_GPUDEVICE environment variable for oarsh allows running the process in the target node with a subset of the GPU devices of the job. Expected format for OAR_USER_GPUDEVICE is a list of , or + or space separated Linux char devices (e.g.: /dev/nvidia0). The GPU devices of the job can be shown with using oarprint (gpudevice property). The oarnodes command also shows the gpudevice resource property for every OAR resource.

EXAMPLES¶

Connecting from within our job, from one node to another one (node23):

 oarsh node-23
    

Connecting to a node (node23) of our job (Id: 4242) from the frontal of the cluster:

 OAR_JOB_ID=4242 oarsh node-23
    

Connecting to a node (node23) of our job that was submitted using a job-key, possibly from anywhere:

 OAR_JOB_KEY_FILE=~/my_key oarsh node-23
    

Same thing but using OpenSSH-like -i option:

 oarsh -i ~/my_key node-23
    

Running a process on a subset of the CPU set of GPU devices of the job:

 OAR_USER_CPUSET=0,7 OAR_USER_GPUDEVICE=/dev/nvidia3 oarsh node-23
    

NOTES¶

oarsh inherits from most of OpenSSH capabilities (e.g. X11 forwarding). However, one feature which oarsh does break is the SSH Agent.

The user's OpenSSH configuration files (files in the ~/.ssh directory) are not used by oarsh.

When a job is running alone on a whole host, it may be possible to simply use ssh instead of oarsh to connect to the host. This require OAR to be configured with the pam_oar_adopt mechanism.

SEE ALSO¶

oarprint(1), oarnodes(1), lstopo(1), ssh(1), oarcp(1), oarsub(1), pam_oar_adopt(8)

COPYRIGHTS¶

 Copyright 2003-2025 Laboratoire d'Informatique de Grenoble (http://www.liglab.fr). This software is licensed under the GNU General Public License Version 2 or above. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.