table of contents
other versions
PROCESS_WORKER_POOL.PY(1) | User Commands | PROCESS_WORKER_POOL.PY(1) |
NAME¶
process_worker_pool.py - create a parsl process worker pool
DESCRIPTION¶
usage: process_worker_pool.py [-h] [-d] [-a ADDRESSES] [-l LOGDIR] [-u UID]
- [-b BLOCK_ID] [-c CORES_PER_WORKER]
- [-m MEM_PER_WORKER] -t TASK_PORT [--max_workers MAX_WORKERS] [-p PREFETCH_CAPACITY] [--hb_period HB_PERIOD] [--hb_threshold HB_THRESHOLD] [--address_probe_timeout ADDRESS_PROBE_TIMEOUT] [--poll POLL] -r RESULT_PORT [--cpu-affinity {none,block,alternating,block-reverse}] [--available-accelerators [AVAILABLE_ACCELERATORS ...]] [--start-method {fork,spawn,thread}]
options:¶
- -h, --help
- show this help message and exit
- -d, --debug
- Count of apps to launch
- -a ADDRESSES, --addresses ADDRESSES
- Comma separated list of addresses at which the interchange could be reached
- -l LOGDIR, --logdir LOGDIR
- Process worker pool log directory
- -u UID, --uid UID
- Unique identifier string for Manager
- -b BLOCK_ID, --block_id BLOCK_ID
- Block identifier for Manager
- -c CORES_PER_WORKER, --cores_per_worker CORES_PER_WORKER
- Number of cores assigned to each worker process. Default=1.0
- -m MEM_PER_WORKER, --mem_per_worker MEM_PER_WORKER
- GB of memory assigned to each worker process. Default=0, no assignment
- -t TASK_PORT, --task_port TASK_PORT
- REQUIRED: Task port for receiving tasks from the interchange
- --max_workers MAX_WORKERS
- Caps the maximum workers that can be launched, default:infinity
- -p PREFETCH_CAPACITY, --prefetch_capacity PREFETCH_CAPACITY
- Number of tasks that can be prefetched to the manager. Default is 0.
- --hb_period HB_PERIOD
- Heartbeat period in seconds. Uses manager default unless set
- --hb_threshold HB_THRESHOLD
- Heartbeat threshold in seconds. Uses manager default unless set
- --address_probe_timeout ADDRESS_PROBE_TIMEOUT
- Timeout to probe for viable address to interchange. Default: 30s
- --poll POLL
- Poll period used in milliseconds
- -r RESULT_PORT, --result_port RESULT_PORT
- REQUIRED: Result port for posting results to the interchange
- --cpu-affinity {none,block,alternating,block-reverse}
- Whether/how workers should control CPU affinity.
- --available-accelerators [AVAILABLE_ACCELERATORS ...]
- Names of available accelerators
- --start-method {fork,spawn,thread}
- Method used to start new worker processes
January 2024 | process_worker_pool.py 2023.11.13+ds |