10.2. Hostfiles

Hostfiles (sometimes called “machine files”) are a combination of two things:

  1. A listing of hosts on which to launch processes.

  2. Optionally, limit the number of processes which can be launched on each host.

Hostfile syntax consists of one node name on each line, optionally including a designated number of “slots”:

# This is a comment line, and will be ignored
node01  slots=10
node13  slots=5

node15
node16
node17  slots=3
...

Blank lines and lines beginning with a # are ignored.

A hostfile may hold at most 64 MiB, and no line in it may be longer than 1 MiB; a larger file is refused with a message saying which limit it met. Reading it may also take no longer than PMIx’s pmix_runas_timeout MCA parameter (10 seconds by default): the DVM master reads the file in a separate process, as the user the file is read for, and gives up on a file whose file system does not answer in that time. The same limits apply to a rankfile and to a sequence file (--map-by seq).

An entry names a node and nothing else. PRRTE reaches every node as the user the DVM runs as, and does not start a daemon as any other user, so an entry may not name an account to reach the node as: user@node and the username= keyword are refused as parse errors, naming the hostfile and the line. So is a rankfile entry naming one.

A node name written with a leading ^ is excluded rather than used:

node01  slots=4
node02  slots=4
node03  slots=4
^node02

Where the hostfile also names nodes, an exclusion takes a node back out of what it named — the file above names node01 and node03. A hostfile given to a job in a running DVM may consist of nothing but exclusions, and then it selects every node of the allocation except the ones it excludes.

A “slot” is the PRRTE term for an allocatable unit where we can launch a process. See the section on definition of the term slot for a longer description of slots.

In the absence of the slot parameter, PRRTE will assign either the number of slots to be the number of CPUs detected on the node or the resource manager-assigned value if operating in the presence of an RM.

Important

If using a resource manager, the user-specified number of slots is capped by the RM-assigned value.

A hostfile given to a job that is being submitted to an already-running DVM selects within the DVM’s allocation: it names the subset of nodes that job may use, and a slots count smaller than the node’s own is the number of slots that job may take there. It says nothing about how big the node is, so it applies to that job alone — the node is back to its allocated size for the next job, which may be someone else’s. Changing the allocation is what --add-hostfile is for.