bgirgis/cpp-thirdparty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Upgrade macos

Browse Source
This commit is contained in:
Bassem Girgis
2025-03-19 08:52:56 -05:00
parent 53580cb51f
commit d29ef955f5
8041 changed files with 1936720 additions and 130842 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
macx64/mpi/openmpi/share/prte/amca-param-sets/example.conf Normal file
View File

@@ -0,0 +1,5 @@
#
# This is an example file illustrating how to setup an Aggregate MCA parameters
# file.
#
#

101
macx64/mpi/openmpi/share/prte/help-cli.txt Normal file
View File

@@ -0,0 +1,101 @@
[bogus section]
This section is not used by PRTE code. But we have to put a RST
section title in this file somewhere, or Sphinx gets unhappy. So we
put it in a section that is ignored by PRTE code.
Hello, world
============
[help]
Help can be requested in three ways:
1. A directive to the cmd. For example, the command
"%s --help" or "%s -h"
will provide the high-level help output giving a general
description of each available option.
2. An option to the help directive. For example, the command
"%s --help foo" or "%s -h foo"
will provide detailed help about the "foo" command line directive.
3. Specifying "help" as an option to the directive of interest. For
example, the command
"%s --foo --help" or "%s --foo -h"
will provide detailed help about the "foo" command line directive.
Note that the "help" and "h" options do not require the dash
prefixes — i.e., the following commands
"%s --foo help" or "%s --foo h"
will also output detailed help for "foo". Thus, directives are
precluded from defining options of "help" and "h".
Note that the last two methods are functionally equivalent and will
return the same output.
[unknown-option]
Help was requested for an unknown option:
Option: %s
Please use the "%s --help" command to obtain a list of all supported
options.
[unrecognized-option]
The help system for %s does not recognize the following:
Option: %s
This might require help from the developers.
[version]
Print version and exit
[verbose]
Enable debug options - number of times specified sets level of
verbosity (e.g., "-vvv" => 3rd level of debug information)
[quiet]
Suppress helpful messages
[parsable]
When used in conjunction with other parameters, output information
(e.g., help messages) is displayed in a machine-parsable format
[parseable]
When used in conjunction with other parameters, the output information
(e.g., help messages) is displayed in a machine-parsable format
[short-no-long]
A short option was provided to %s and recognized by the parser, but
has no matching long option:
Short option: %s
This might require help from the developers.
[unregistered-option]
An unrecognized option was included on the %s command line:
Option: %s
Please use the "%s --help" command to obtain a list of all supported
options.

68
macx64/mpi/openmpi/share/prte/help-dash-host.txt Normal file
View File

@@ -0,0 +1,68 @@
[bogus section]
This section is not used by PRTE code. But we have to put a RST
section title in this file somewhere, or Sphinx gets unhappy. So we
put it in a section that is ignored by PRTE code.
Hello, world
============
[not-all-mapped-alloc]
At least one of the requested hosts is not included in the current
allocation.
Missing requested host: %s
Please check your allocation or your request.
[dash-host:relative-syntax]
A relative host was specified, but no prior allocation has been made.
Thus, there is no way to determine the proper host to be used.
--host: %s
Re-run this command with "--help hosts" for further information.
[dash-host:relative-node-not-found]
A relative host was specified, but was not found. The requested host
was specified with "--host" as:
Index: %d
Syntax given: %s
Re-run this command with "--help hosts" for further information.
[dash-host:relative-node-out-of-bounds]
A relative host was specified, but the index given is beyond the
number of hosts in the current allocation:
Index: %d
#hosts: %d
You could obtain a larger allocation or reduce the relative host
index.
Re-run this command with "--help hosts" for further information.
[dash-host:invalid-relative-node-syntax]
A relative host was improperly specified — the value provided was.
--host: %s
You may have forgotten to preface a node with "N" or "n", or used the
"e" or "E" to indicate empty nodes.
Re-run this command with "--help hosts" for further information.
[dash-host:not-enough-empty]
The requested number of empty hosts was not available — the system was
short by %d hosts. Please recheck your allocation.
Re-run this command with "--help hosts" for further information.

117
macx64/mpi/openmpi/share/prte/help-errmgr-base.txt Normal file
View File

@@ -0,0 +1,117 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2014-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2017 IBM Corporation. All rights reserved.
# Copyright (c) 2018 Research Organization for Information Science
# and Technology (RIST). All rights reserved.
# Copyright (c) 2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2022-2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
#
[undeliverable-msg]
An attempt was made to send a message to a process whose
address is unknown:
Sender: %s
From node: %s
Recipient: %s
On node: %s
The message could not be delivered, and we are aborting.
#
[failed-daemon-launch]
%s was unable to reliably start one or more daemons.
This usually is caused by:
* the inability to resolve specified hostnames - please check
above for any output regarding this possibility
* not finding the required libraries and/or binaries on
one or more nodes. Please check your PATH and LD_LIBRARY_PATH
settings, or configure OMPI with --enable-prun-prefix-by-default
* lack of authority to execute on one or more specified nodes.
Please verify your allocation and authorities.
* the inability to write startup files into /tmp (--tmpdir/prte_tmpdir_base).
Please check with your sys admin to determine the correct location to use.
* compilation of the orted with dynamic libraries when static are required
(e.g., on Cray). Please check your configure cmd line and consider using
one of the contrib/platform definitions for your system type.
* an inability to create a connection back to mpirun due to a
lack of common network interfaces and/or no route found between
them. Please check network connectivity (including firewalls
and network routing requirements).
#
[failed-daemon]
An PRTE daemon has unexpectedly failed after launch and before
communicating back to mpirun. This could be caused by a number
of factors, including an inability to create a connection back
to mpirun due to a lack of common network interfaces and/or no
route found between them. Please check network connectivity
(including firewalls and network routing requirements).
#
[node-died]
PRTE has lost communication with a remote daemon.
HNP daemon : %s on node %s
Remote daemon: %s on node %s
This is usually due to either a failure of the TCP network
connection to the node, or possibly an internal failure of
the daemon itself. We cannot recover from this failure, and
therefore will terminate the job.
#
[no-path]
PRTE does not know how to route a message to the specified daemon
located on the indicated node:
my node: %s
target node: %s
This is usually an internal programming error that should be
reported to the developers. In the meantime, a workaround may
be to set the MCA param routed=direct on the command line or
in your environment. We apologize for the problem.
#
[no-connect]
PRTE is unable to establish a communication connection to the
specified daemon located on the indicated node:
my node: %s
target node: %s
This is usually due to a lack of common network interfaces and/or
no route found between them. Please check network connectivity (including
firewalls and network routing requirements). If these look okay,
then it could be an internal programming error that should be
reported to the developers. In the meantime, a workaround may
be to set the MCA param routed=direct on the command line or
in your environment.
#
[simple-message]
An internal error has occurred in PRTE:
%s
This is something that should be reported to the developers.
#
[normal-termination-but]
%s job %s terminated normally, but %d %s. Per user-direction, the job has been aborted.

99
macx64/mpi/openmpi/share/prte/help-ess-base.txt Normal file
View File

@@ -0,0 +1,99 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2017-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English general help file for the SDS base.
#
[ess-base:execv-error]
The singleton application was not able to find the executable "orted" in
your PATH or in the directory where PRTE was initially installed,
and therefore cannot continue.
For reference, we tried the following command:
%s
and got the error %s.
This could mean that your PATH or executable name is wrong, or that you do not
have the necessary permissions. Please ensure that the executable is able to be
found and executed as it is required for singleton operations.
[ess-base:cannot-forward]
The system does not support trapping and forwarding of the
specified signal:
signal: %s
param: %s
Please remove that signal from the ess_base_forward_signals MCA parameter.
[ess-base:unknown-signal]
The following signal was included in the ess_base_forward_signals
MCA parameter:
signal: %s
param: %s
This is not a recognized signal value. Please fix or remove it.
#
[slurm-error]
The application appears to have been direct launched using "srun",
but OMPI was not built with SLURM's PMI support and therefore cannot
execute. There are several options for building PMI support under
SLURM, depending upon the SLURM version you are using:
version 16.05 or later: you can use SLURM's PMIx support. This
requires that you configure and build SLURM --with-pmix.
Versions earlier than 16.05: you must use either SLURM's PMI-1 or
PMI-2 support. SLURM builds PMI-1 by default, or you can manually
install PMI-2. You must then build PRTE using --with-pmi pointing
to the SLURM PMI library location.
Please configure as appropriate and try again.
#
[slurm-error2]
The application appears to have been direct launched using "srun",
but OMPI was not built with SLURM support. This usually happens
when OMPI was not configured --with-slurm and we weren't able
to discover a SLURM installation in the usual places.
Please configure as appropriate and try again.
#
[alps-error]
The application appears to have been direct launched using "aprun",
but OMPI was not built with ALPS PMI support and therefore cannot
execute. You must build PRTE using --with-pmi pointing
to the ALPS PMI library location.
Please configure as appropriate and try again.
#
[alps-error2]
The application appears to have been direct launched using "aprun",
but OMPI was not built with ALPS support. This usually happens
when OMPI was not configured --with-alps and we weren't able
to discover an ALPS installation in the usual places.
Please configure as appropriate and try again.
#
[legacy-tool]
We no longer support non-PMIx-based tools, and require a
minimum level of PMIx v2.0.
Please update the tool and/or the PMIx version you are using.

196
macx64/mpi/openmpi/share/prte/help-hostfile.txt Normal file
View File

@@ -0,0 +1,196 @@
[bogus section]
This section is not used by PRTE code. But we have to put a RST
section title in this file somewhere, or Sphinx gets unhappy. So we
put it in a section that is ignored by PRTE code.
Hello, world
============
[no-hostfile]
PRTE was unable to open the hostfile:
%s
Check to make sure the path and filename are correct.
[port]
PRTE detected a bad parameter in the hostfile:
%s
The port parameter is less than 0:
port=%d
[slots]
PRTE detected a bad parameter in the hostfile:
%s
The slots parameter is less than 0:
slots=%d
[max_slots]
PRTE detected a bad parameter in the hostfile:
%s
The max_slots parameter is less than 0:
max_slots=%d
[max_slots_lt]
PRTE detected a bad parameter in the hostfile:
%s
The max_slots parameter is less than the slots parameter:
slots=%d
max_slots=%d
[parse_error_string]
PRTE detected a parse error in the hostfile:
%s
It occured on line number %d on token %d:
%s
[parse_error_int]
PRTE detected a parse error in the hostfile:
%s
It occured on line number %d on token %d:
%d
[parse_error]
PRTE detected a parse error in the hostfile:
%s
It occured on line number %d on token %d.
[not-all-mapped-alloc]
Some of the requested hosts are not included in the current
allocation.
The requested hosts were in this hostfile:
%s
Please verify that you have specified the allocated resources properly
in the provided hostfile.
[hostfile:relative-syntax]
A relative host was specified, but no prior allocation has been made.
Thus, there is no way to determine the proper host to be used.
hostfile entry: %s
Re-run this command with "--help hosts" for further information.
[hostfile:relative-node-not-found]
A relative host was specified, but was not found. The requested host
was specified as:
Index: %d
Syntax given: %s
This is most likely due to the relative index being out of bounds. You
could obtain a larger allocation or reduce the relative host index.
Re-run this command with "--help hosts" for further information.
[hostfile:invalid-relative-node-syntax]
A relative host was improperly specified — the value provided was.
hostfile entry: %s
You may have forgotten to preface a node with "N" or "n", or used the
"e" or "E" to indicate empty nodes.
Re-run this command with "--help hosts" for further information.
[hostfile:not-enough-empty]
The requested number of empty hosts was not available — the system was
short by %d hosts. Please recheck your allocation.
Re-run this command with "--help hosts" for further information.
[boards]
PRTE detected a bad parameter in the hostfile:
%s
The boards parameter is less than 0:
boards=%d
[sockets]
PRTE detected a bad parameter in the hostfile:
%s
The packages parameter is less than 0:
packages=%d
[cores]
PRTE detected a bad parameter in the hostfile:
%s
The cores parameter is less than 0:
cores=%d
[hostfile:extra-node-not-found]
A hostfile was provided that contains at least one node not present in
the allocation:
hostfile: %s
node: %s
If you are operating in a resource-managed environment, then only
nodes that are in the allocation can be used in the hostfile. You may
find relative node syntax to be a useful alternative to specifying
absolute node names; re-run this command with "--help hosts" for
further information.
[slots-given]
A hostfile was provided that contains multiple definitions of the slot
count for at least one node:
hostfile: %s
node: %s
You can either list a node multiple times, once for each slot, or you
can provide a single line that contains "slot=N". Mixing the two
methods is not supported.
Please correct the hostfile and try again.

32
macx64/mpi/openmpi/share/prte/help-iof-base.txt Normal file
View File

@@ -0,0 +1,32 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2014-2019 Intel, Inc. All rights reserved.
# Copyright (c) 2017 IBM Corporation. All rights reserved.
# Copyright (c) 2018 Research Organization for Information Science
# and Technology (RIST). All rights reserved.
# Copyright (c) 2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
#
[unrecognized-directive]
The following directive was given to "--%s":
Directive: %s
This directive is not recognized. Please check your spelling
and/or use the "--help" option to find the supported values.
#

24
macx64/mpi/openmpi/share/prte/help-oob-base.txt Normal file
View File

@@ -0,0 +1,24 @@
# -*- text -*-
#
# Copyright (c) 2004-2006 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2006 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2014-2019 Intel, Inc. All rights reserved.
# Copyright (c) 2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
#
[no-interfaces-avail]
No network interfaces were found for out-of-band communications. We require
at least one available network for out-of-band messaging.

139
macx64/mpi/openmpi/share/prte/help-oob-tcp.txt Normal file
View File

@@ -0,0 +1,139 @@
# -*- text -*-
#
# Copyright (c) 2004-2006 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2006 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2014-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2015-2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
[static-and-dynamic]
Both static and dynamic port ranges were specified for the
out-of-band (OOB) communication subsystem:
Static ports: %s
Dynamic ports: %s
Only one can be specified. Please choose either static or
dynamic ports and try again.
#
[include-exclude]
Both TCP interface include and exclude lists were specified:
Include: %s
Exclude: %s
Only one of these can be given.
#
[not-parseable]
The specified network is not parseable. Since we cannot determine
your desired intent, we cannot establish a TCP socket for out-of-band
communications and will therefore abort. Please correct the network
specification and retry.
#
[no-included-found]
None of the TCP networks specified to be included for out-of-band communications
could be found:
Value given: %s
Please revise the specification and try again.
#
[excluded-all]
The specified list of networks to be excluded for TCP out-of-band communications
resulted in no networks being available:
Value given: %s
Please revise the specification and try again.
#
[invalid if_inexclude]
WARNING: An invalid value was given for oob_tcp_if_%s. This
value will be ignored.
Local host: %s
Value: %s
Message: %s
#
[authent-fail]
An attempt was made to make a TCP connection between two hosts:
Initiating host: %s
Receiving host: %s
Unfortunately, the connection was refused due to a failure to
authenticate. This is usually caused by a mis-match between
the security domains of the two hosts - e.g., one might be
using Munge while the other is not. This can typically be
resolved by specifying the desired security method. For
example, adding "--mca sec basic" to your command line.
#
[accept failed]
WARNING: The accept(3) system call failed on a TCP socket. While this
should generally never happen on a well-configured HPC system, the
most common causes when it does occur are:
* The process ran out of file descriptors
* The operating system ran out of file descriptors
* The operating system ran out of memory
Your PRTE job will likely hang until the failure resason is fixed
(e.g., more file descriptors and/or memory becomes available), and may
eventually timeout / abort.
Local host: %s
Errno: %d (%s)
Probable cause: %s
#
[privilege failure]
An attempt was made to initiate a TCP connection from an
unprivileged source while we are operating at privileged
levels.
Local host: %s
Listening port: %d
Remote host: %s
Remote port: %d
The connection was rejected.
#
[static-fwd]
Static ports were requested while prte_fwd_mpirun_port was set.
Both options cannot be simultaneously set. Please either set
prte_fwd_mpirun_port=false or remove any static port directives.
#
[version mismatch]
PRTE detected a mismatch in versions between two processes. This
typically means that you executed "mpirun" (or "mpiexec") from one
version of PRTE on on node, but your default path on one of the
other nodes upon which you launched found a different version of Open
MPI.
PRTE only supports running exactly the same version between all
processes in a single job.
This will almost certainly cause unpredictable behavior, and may end
up aborting your job.
Local host: %s
Local process name: %s
Local PRTE version: %s
Peer host: %s
Peer process name: %s
Peer PRTE version: %s
#
[no-listeners]
No sockets were able to be opened on the available protocols
(IPv4 and/or IPv6). Please check your network and retry.

177
macx64/mpi/openmpi/share/prte/help-plm-base.txt Normal file
View File

@@ -0,0 +1,177 @@
# -*- text -*-
#
# Copyright (c) 2004-2006 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2006 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2015-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2021-2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
[no-available-pls]
No available launching agents were found.
This is an unusual error; it means that PRTE was unable to find
any mechanism to launch proceses, and therefore is unable to start the
process(es) required by your application.
#
[daemon-no-assigned-node]
A daemon has no recorded node:
Daemon: %s
Reported from nodename: %s
This usually indicates a difference between the names of nodes in the
allocation versus what is returned on the node by get_hostname.
#
[daemon-died-no-signal]
A daemon died unexpectedly with status %d while attempting
to launch so we are aborting.
There may be more information reported by the environment (see above).
This may be because the daemon was unable to find all the needed shared
libraries on the remote node. You may set your LD_LIBRARY_PATH to have the
location of the shared libraries on the remote nodes and this will
automatically be forwarded to the remote nodes.
#
[daemon-died-signal-core]
A daemon died unexpectedly on signal %d (with core) while
attempting to launch so we are aborting.
There may be more information reported by the environment (see above).
This may be because the daemon was unable to find all the needed shared
libraries on the remote node. You may set your LD_LIBRARY_PATH to have the
location of the shared libraries on the remote nodes and this will
automatically be forwarded to the remote nodes.
#
[daemon-died-signal]
A daemon died unexpectedly on signal %d while attempting to
launch so we are aborting.
There may be more information reported by the environment (see above).
This may be because the daemon was unable to find all the needed shared
libraries on the remote node. You may set your LD_LIBRARY_PATH to have the
location of the shared libraries on the remote nodes and this will
automatically be forwarded to the remote nodes.
#
[incomplete-exit-cmd]
One or more daemons could not be ordered to exit. This can be caused by a
number of rather rare problems, but typically is caused by a daemon having
died due to the failure of a node or its communications. This could result
in an incomplete cleanup on the affected nodes. Please see below for a list
of nodes which may require additional cleanup.
We are truly sorry for the inconvenience.
#
[incomplete-kill-procs-cmd]
One or more daemons could not be ordered to kill their local processes.
This can be caused by a number of rather rare problems, but typically
is caused by a daemon having died due to the failure of a node or its
communications. This could result in an incomplete cleanup on the affected
nodes. Additional information may be available below.
We are truly sorry for the inconvenience.
#
[stdin-target-out-of-range]
The requested stdin target is out of range for this job - it points
to a process rank that is greater than the number of processes in the
job.
Specified target: %s
Number of procs: %s
This could be caused by specifying a negative number for the stdin
target, or by mistyping the desired rank. Remember that MPI ranks begin
with 0, not 1.
Please correct the cmd line and try again.
#
[too-many-hosts]
A call was made to launch a local slave process, but more than one
target host was provided. Currently, each launch of a local slave
can only be to a single host. To launch slaves on multiple hosts,
you must issue one call/host.
Num hosts specified: %d
#
[abs-path-reqd]
A call was made to launch a local slave process that requested the
binaries be pre-positioned on the remote host. However, an absolute
path to the target directory was either not specified, or was provided
in a relative path format.
Path provided: %s
The path to the target directory must be given as an absolute path. The
target directory does NOT need to exist - the path to the target will
be created, if required.
#
[exec-not-found]
A call was made to launch a local slave process, but the specified
executable could not be found:
Exec: %s
Please either specify an absolute path to the executable, or check
that the executable is in your PATH.
#
[cp-not-found]
A call was made to launch a local slave process that requested the
binaries be pre-positioned on the remote host. However, we could not
find the %s command in your path.
Please check that %s is in your PATH and try again.
#
[file-not-found]
A call was made to launch a local slave process that requested pre-positioning
of one or more files, but the specified files could not be found:
File: %s
Source directory: %s
Please either specify an absolute path to the file, or check
that the file is in your current working directory...and ensure
that you have read permissions on the file.
#
[deprecated-amca]
You're using the --am option. Please be advised that this option is deprecated;
you should use --tune instead. The --tune option allows one to specify mca
parameters as well as environment variables from within a file using the same
command line syntax (e.g. -mca var val -mca var "val" -x var=val -x var).
The --am option will be removed in a future release.
#
[no-oob]
A call was made to launch additional processes, but this process has
no active out-of-band transports and therefore cannot execute this call.
Please check to see if you have the "oob" MCA parameter set and ensure
that it is either unset or at least includes the tcp transport.
#
[multi-endian]
PRTE does not currently support multi-endian operations. We have
detected that the following node differs in endianness:
Nodename: %s
Endian: %s
Local endian: %s
Please correct the situation and try again.

138
macx64/mpi/openmpi/share/prte/help-plm-slurm.txt Normal file
View File

@@ -0,0 +1,138 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2014-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2022-2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
[multiple-prefixes]
The SLURM process starter for PRTE does not support multiple
different --prefix options to mpirun. You can specify at most one
unique value for the --prefix option (in any of the application
contexts); it will be applied to all the application contexts of your
parallel job.
Put simply, you must have PRTE installed in the same location on
all of your SLURM nodes.
Multiple different --prefix options were specified to mpirun. This is
a fatal error for the SLURM process starter in PRTE.
The first two prefix values supplied were:
%s
and %s
#
[no-hosts-in-list]
The SLURM process starter for PRTE didn't find any hosts in
the map for this application. This can be caused by a lack of
an allocation, or by an error in the PRTE code. Please check
to ensure you have a SLURM allocation. If you do, then please pass
the error to the PRTE user's mailing list for assistance.
#
[no-local-slave-support]
A call was made to launch a local slave process, but no support
is available for doing so. Launching a local slave requires support
for either rsh or ssh on the backend nodes where MPI processes
are running.
Please consult with your system administrator about obtaining
such support.
#
[no-srun]
The SLURM process starter for OpenMPI was unable to locate a
usable "srun" command in its path. Please check your path
and try again.
#
[ancient-version]
The Slurm process starter requires a minimum Slurm version level
of v17.11. The detected version being used here is:
Major: %d
Minor: %d
SchedMD strongly recommends that you upgrade your Slurm version.
Versions of Slurm prior to 21.08.8 and 20.11.9 have been found
to have a security issue and are not supported - therefore, it
is recommended that you upgrade to a level at or above those
versions.
It is recognized that upgrades can take some time to complete.
In the interim, you can execute your job by ignoring the Slurm
process starter via the following MCA parameter:
Environment: PRTE_MCA_plm=^slurm
Cmd line: --prtemca plm ^slurm
Default MCA param file: plm = ^slurm
This will result in use of the ssh process starter. This will have
no impact on your application, but will result in any accounting
being done solely at the allocation level instead of per-job.
#
[custom-args-in-env]
The Slurm process starter for PRTE detected the presence of an MCA
parameter in the environment that assigns custom command line arguments
to the `srun` command used to start PRTE's daemons on remote nodes:
Paramater value: %s
This warning is provided to alert you (the user) to a perhaps
unintentional setting of command line arguments, or the unseen
overriding of your intended arguments by Slurm.
Background: Starting with Slurm version 23.11, a command line argument
(`--external-launcher`) was added to `srun` to indicate that the
command was being initiated from within a third-party launcher (e.g.,
`prte` or `prterun`). This allows Slurm to essentially freely modify
the `srun` command line while retaining a backward compatibility
capability when explicitly told to use it. Notably, the Slurm
environment does this by automatically setting the
PRTE_MCA_plm_slurm_args environment variable to pass in its own
command line arguments. This has the side effect of overriding most
user- or system-level settings. Note that arguments passed on the
PRTE command line will override any Slurm setting of the
PRTE_MCA_plm_slurm_args environment variable, but with potentially
undesirable side effects if newer versions of `srun` misinterpret or
fail to understand the user-specified arguments.
If the setting of the MCA parameter was intentional, or if the
parameter looks acceptable to you, then please set the following
MCA parameter to disable this warning:
Environment: PRTE_MCA_plm_slurm_disable_warning=true
Cmd line: --prtemca plm_slurm_disable_warning 1
Default MCA param file: plm_slurm_disable_warning = true
If you did not intentionally set the identified command line
arguments and do not wish them to be used, then set the
following MCA param to have them ignored:
Environment: PRTE_MCA_plm_slurm_ignore_args=true
Cmd line: --prtemca plm_slurm_ignore_args 1
Default MCA param file: plm_slurm_ignore_args = true
Note that if you wish to provide custom `srun` command line
arguments and are finding them being overridden by Slurm, you
can ensure that your values are used by setting them with the
following param:
Environment: PRTE_MCA_plm_slurm_force_args=foo
Cmd line: --prtemca plm_slurm_force_args foo
Default MCA param file: plm_slurm_force_args = foo
Note that you may need to add the `--external-launcher` option
to your provided args to ensure that `srun` properly functions
if you are using a relatively recent release of Slurm.

98
macx64/mpi/openmpi/share/prte/help-plm-ssh.txt Normal file
View File

@@ -0,0 +1,98 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2015-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2020-2021 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English general help file for PRTE's prun.
#
[agent-not-found]
The value of the MCA parameter "plm_ssh_agent" was set to a path
that could not be found:
plm_ssh_agent: %s
Please either unset the parameter, or check that the path is correct
#
[no-local-orted]
The ssh PLS component was not able to find the executable "orted" in
your PATH or in the directory where PRTE was initially installed,
and therefore cannot continue.
For reference, your current PATH is:
%s
We also looked for orted in the following directory:
%s
[multiple-prefixes]
Specified multiple application contexts using different
settings for --prefix. Care should be taken, that corresponding
processes are mapped to different nodes. Having multiple prefixes
per node is not allowed.
The previously set prefix was
%s
the prefix to be set overriding:
%s
[concurrency-less-than-zero]
The value of the MCA parameter "pls_ssh_num_concurrent" is less than
or equal to zero (%d). This parameter is used to determine how many
remote agents (typically ssh) to invoke concurrently while launching
parallel jobs.
This value has automatically be reset to 1; processing will continue.
[deadlock-params]
The ssh launcher has been given a number of %d concurrent daemons to
launch and is in a debug-daemons option. However, the total number of
daemons to launch (%d) is greater than this value. This is a scenario that
will cause the system to deadlock.
To avoid deadlock, either increase the number of concurrent daemons, or
remove the debug-daemons flag.
[unknown-user]
The user (%d) is unknown to the system (i.e. there is no corresponding
entry in the password file). Please contact your system administrator
for a fix.
#
[cannot-resolve-shell-with-prefix]
The ssh launcher has been given a prefix to use, but could not determine
the type of remote shell being used on the remote node. This is a fatal
error as we cannot determine how to construct the cmd line to set your
remote LD_LIBRARY_PATH and PATH environmental variables.
The prefix we were given are:
prte_prefix: %s
prefix_dir: %s
#
[cmd-line-too-long]
The cmd line to launch remote daemons is too long:
Length: %d
Max length: %d
Consider setting -mca plm_ssh_pass_environ_mca_params 0 to
avoid including any environmentally set MCA parameters on the
command line.

49
macx64/mpi/openmpi/share/prte/help-prte-filem-raw.txt Normal file
View File

@@ -0,0 +1,49 @@
-*- text -*-
#
# Copyright (c) 2004-2007 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2019 Intel, Inc. All rights reserved.
# Copyright (c) 2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English general help file for PRTE FileM framework.
#
[prte-filem-rsh:get-file-exists]
WARNING: Could not preload specified file: File already exists.
Fileset: %s
Host: %s
Will continue attempting to launch the process.
[prte-filem-rsh:put-file-not-exist]
WARNING: Could not preload specified file: File does not exist.
Fileset: %s
Host: %s
Will continue attempting to launch the process.
[prte-filem-rsh:remote-get-failed]
WARNING: Remote peer (%s) failed to preload a file.
Exit Status: %d
Local File: %s
Remote File: %s
Command:
%s
Will continue attempting to launch the process(es).

105
macx64/mpi/openmpi/share/prte/help-prte-hwloc-base.txt Normal file
View File

@@ -0,0 +1,105 @@
[bogus section]
This section is not used by PRTE code. But we have to put a RST
section title in this file somewhere, or Sphinx gets unhappy. So we
put it in a section that is ignored by PRTE code.
Hello, world
============
[mbind failure]
PRTE failed to bind internal memory to a specific NUMA node. This
message will only be reported at most once per process.
Local host: %s
PID: %d
File: %s:%d
Message: %s
Severity: %s
[invalid binding_policy]
The specified %s policy is not recognized:
Policy: %s
Please check for a typo or ensure that the option is a supported one.
[redefining-policy]
Conflicting directives for binding policy are causing the policy to be
redefined:
New policy: %s
Prior policy: %s
Please check that only one policy is defined.
[deprecated]
The following command line option and corresponding MCA parameter have
been deprecated and replaced as follows:
Command line option:
Deprecated: %s
Replacement: %s
Equivalent MCA parameter:
Deprecated: %s
Replacement: %s
The deprecated forms *will* disappear in a future version of PRTE.
Please update to the new syntax.
[obj-idx-failed]
PRTE failed to find a cache of a specified type. This is a highly
unusual error; it may indicate a system configuration error. This
additional information may be of help:
Message: %s
Cache level: %d
[missing-cpulist]
The following binding modifier was given but lacks a required value:
Modifier: %s
Please check for a typo or ensure that the option is a supported one.
[bad-processor-type]
The default CPU list contains an invalid modifier:
Default CPU list: %s
Modifier: %s
The modifier should indicate the type of CPU being used in the list —
the only valid values are HWTCPUS (for use of hwthreads as independent
CPUs) or CORECPUS (for use of cores as independent CPUs). Please
correct the input.
[unrecognized-modifier]
The binding request contains an unrecognized modifier:
Request: %s
Please check your request and try again.
[bind-upwards]
Binding is performed to the first available specified object type
within the object where the process was mapped. In other words,
binding can only be done to the mapped object or to a resource located
beneath that object.
The specified binding lies above the mapping object type:
Mapping level: %s
Binding level: %s
Please correct the map/bind directives and try again.

140
macx64/mpi/openmpi/share/prte/help-prte-info.txt Normal file
View File

@@ -0,0 +1,140 @@
[bogus section]
This section is not used by PRTE code. But we have to put a RST
section title in this file somewhere, or Sphinx gets unhappy. So we
put it in a section that is ignored by PRTE code.
Hello, world
============
[usage]
%s (%s) %s
Usage: "%s [OPTION]..."
Provide detailed information on your PRRTE installation.
The following list of command line options are available. Note that
more detailed help for any option can be obtained by adding that
option to the help request as "--help <option>".
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "-a" | "--all" | Show all configuration options and MCA |
| | parameters |
+----------------------+-----------------------------------------------+
| "--arch" | Show architecture on which PRRTE was compiled |
+----------------------+-----------------------------------------------+
| "-c" | "--config" | Show configuration options |
+----------------------+-----------------------------------------------+
| "-h" | "--help" | This help message |
+----------------------+-----------------------------------------------+
| "--hostname" | Show the hostname on which PRRTE was |
| | configured and built |
+----------------------+-----------------------------------------------+
| "--internal" | Show internal MCA parameters (not meant to be |
| | modified by users) |
+----------------------+-----------------------------------------------+
| "--param <framework | Show MCA parameters. The first parameter is |
| >:<component1>,<com | the framework (or the keyword "all"); the |
| ponent2>" | second parameter is a comma-delimited list of |
| | specific component names (if only |
| | "<framework>" is given, then all components |
| | will be reported). |
+----------------------+-----------------------------------------------+
| "--path <type>" | Show paths with which PRRTE was configured. |
| | Accepts the following parameters: "prefix", |
| | "bindir", "libdir", "incdir", "mandir", |
| | "pkglibdir", "sysconfdir". |
+----------------------+-----------------------------------------------+
| "--show-version | Show version of PRRTE or a component. The |
| <type>:<part>" | first parameter can be the keywords "prte", |
| | "all", or a framework name (indicating all |
| | components in a framework), or a |
| | "framework:component" string (indicating a |
| | specific component). The second parameter |
| | can be one of: "full", "major", "minor", |
| | "release", "greek", "git". |
+----------------------+-----------------------------------------------+
| "-V" | "--version" | Print version and exit |
+----------------------+-----------------------------------------------+
Report bugs to %s
[param]
Syntax: "--param <arg0> <arg1>"
Show MCA parameters. The first parameter is the framework (or the
keyword "all"); the second parameter is the specific component name
(or the keyword "all").
[internal]
Syntax: "--internal"
Show internal MCA parameters (i.e., parameters not meant to be
modified by users)
[path]
Syntax: "--path <arg0>"
Show the paths with which PRRTE was configured. Accepts the following
parameters: "prefix", "bindir", "libdir", "incdir", "mandir",
"pkglibdir", "sysconfdir".
[arch]
Syntax: "--arch" Show architecture on which PRRTE was compiled
[config]
Syntax: "-c" or "--config"
Show configuration options used to configure PRRTE
[hostname]
Syntax: "--hostname"
Show the hostname upon which PRRTE was configured and built
[all]
Syntax: "-a" or "--all"
Show all configuration options and MCA parameters
[lib-call-fail]
A library call unexpectedly failed. This is a terminal error; please
show this message to a PRTE wizard:
Library call: %s Source file: %s Source line number: %d
Aborting...
[developer warning: field too long]
DEVELOPER WARNING: A field in prte_info output is too long and will
appear poorly in the prettyprint output.
Value: "%s" Max length: %d
[not-found]
The specified framework could not be found:
Framework: %s
This could be due to a misspelling of the framework name, or because
support for that framework was not configured into this version of
PRTE. Please see
prte_info --config
for a full report of how PRTE was configured.

104
macx64/mpi/openmpi/share/prte/help-prte-odls-base.txt Normal file
View File

@@ -0,0 +1,104 @@
# -*- text -*-
#
# Copyright (c) 2004-2007 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2010-2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2014 Research Organization for Information Science
# and Technology (RIST). All rights reserved.
# Copyright (c) 2017-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English general help file for PRTE's ODLS Framework
#
[prte-odls-base:could-not-kill]
WARNING: A process refused to die despite all the efforts!
This process may still be running and/or consuming resources.
Host: %s
PID: %d
[prte-odls-base:could-not-preload-binary]
WARNING: Could not preload the binary file.
Binary: %s
Will continue attempting to launch the process.
[prte-odls-base:could-not-preload-files]
WARNING: Could not preload the files specified.
Fileset: %s
Will continue attempting to launch the process.
[prte-odls-base:could-not-preload]
WARNING: Could not preload the requested files and directories.
Binary : %s
Fileset: %s
Will continue attempting to launch the process.
#
[prte-odls-base:xterm-rank-out-of-bounds]
The xterm option was asked to display a rank that is larger
than the number of procs in the job:
Node: %s
Rank: %d
Num procs: %d
Note that ranks start with 0, not 1, and must be specified
accordingly.
#
[prte-odls-base:xterm-neg-rank]
The xterm option was asked to display a rank that is negative:
Rank: %d
Num procs: %d
Note that ranks start with 0, not 1, and must be specified
accordingly.
#
[prte-odls-base:show-bindings]
System has detected external process binding to cores %04lx.
#
[warn not bound]
A request to bind the processes to a %s was made, but the operation
resulted in the processes being unbound. This was most likely caused
by the following:
%s
This is only a warning that can be suppressed in the future by
setting the odls_warn_if_not_bound MCA parameter to 0. Execution
will continue.
Local host: %s
Application name: %s
Action requested: %s %s
#
[error not bound]
A request to bind the processes to a %s was made, but the operation
resulted in the processes being unbound. This was most likely caused
by the following:
%s
This is an error; your job will now abort.
Local host: %s
Application name: %s
Action requested: %s %s
#
[prte-odls-base:fork-agent-not-found]
The specified fork agent was not found:
Node: %s
Fork agent: %s
The application cannot be launched.

141
macx64/mpi/openmpi/share/prte/help-prte-odls-default.txt Normal file
View File

@@ -0,0 +1,141 @@
# -*- text -*-
#
# Copyright (c) 2004-2007 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2009 Sun Microsystems, Inc. All rights reserved.
# Copyright (c) 2010-2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2017-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is a US/English help file.
#
[execve error]
PRTE tried to fork a new process via the "execve" system call but
failed. PRTE checks many things before attempting to launch a
child process, but nothing is perfect. This error may be indicative
of another problem on the target host, or even something as silly as
having specified a directory for your application. Your job will now
abort.
Local host: %s
Working dir: %s
Application name: %s
Error: %s
#
[binding not supported]
PRTE tried to bind a new process, but process binding is not
supported on the host where it was launched. The process was killed
without launching the target application. Your job will now abort.
Local host: %s
Application name: %s
#
[binding generic error]
PRTE tried to bind a new process, but something went wrong. The
process was killed without launching the target application. Your job
will now abort.
Local host: %s
Application name: %s
Error message: %s
Location: %s:%d
#
[bound to everything]
PRTE tried to bind a new process to a specific set of processors,
but ended up binding it to *all* processors. This means that the new
process is effectively unbound.
This is only a warning -- your job will continue. You can suppress
this warning in the future by setting the odls_warn_if_not_bound MCA
parameter to 0.
Local host: %s
Application name: %s
Location: %s:%d
#
[slot list and paffinity_alone]
PRTE detected that both a slot list was specified and the MCA
parameter "paffinity_alone" was set to true. Only one of these can be
used at a time. Your job will now abort.
Local host: %s
Application name: %s
#
[iof setup failed]
PRTE tried to launch a child process but the "IOF child setup"
failed. This should not happen. Your job will now abort.
Local host: %s
Application name: %s
#
[not bound]
WARNING: PRTE tried to bind a process but failed. This is a
warning only; your job will continue.
Local host: %s
Application name: %s
Error message: %s
Location: %s:%d
#
[syscall fail]
A system call failed that should not have. In this particular case,
a warning or error message was not displayed that should have been.
Your job may behave unpredictably after this, or abort.
Local host: %s
Application name: %s
Function: %s
Location: %s:%d
#
[memory not bound]
WARNING: PRTE tried to bind a process but failed. This is a
warning only; your job will continue, though performance may
be degraded.
Local host: %s
Application name: %s
Error message: %s
Location: %s:%d
#
[memory binding error]
PRTE tried to bind memory for a new process but something went
wrong. The process was killed without launching the target
application. Your job will now abort.
Local host: %s
Application name: %s
Error message: %s
Location: %s:%d
#
[set limit]
Error message received from:
Local host: %s
Application name: %s
Location: %s:%d
Message:
%s
#
[incorrectly-bound]
WARNING: PRTE incorrectly bound a process to the daemon's cores.
This is a warning only; your job will continue.
Local host: %s
Application name: %s
Location: %s:%d

831
macx64/mpi/openmpi/share/prte/help-prte-rmaps-base.txt Normal file
View File

@@ -0,0 +1,831 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2011-2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2011 Los Alamos National Security, LLC.
# All rights reserved.
# Copyright (c) 2014-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2021-2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English general help file for PRRTE's prun.
#
[prte-rmaps-base:alloc-error]
There are not enough slots available in the system to satisfy the %d
slots that were requested by the application:
%s
Either request fewer procs for your application, or make more slots
available for use.
A "slot" is the PRRTE term for an allocatable unit where we can
launch a process. The number of slots available are defined by the
environment in which PRRTE processes are run:
1. Hostfile, via "slots=N" clauses (N defaults to number of
processor cores if not provided)
2. The --host command line parameter, via a ":N" suffix on the
hostname (N defaults to 1 if not provided)
3. Resource manager (e.g., SLURM, PBS/Torque, LSF, etc.)
4. If none of a hostfile, the --host command line parameter, or an
RM is present, PRRTE defaults to the number of processor cores
In all the above cases, if you want PRRTE to default to the number
of hardware threads instead of the number of processor cores, use the
--use-hwthread-cpus option.
Alternatively, you can use the --map-by :OVERSUBSCRIBE option to ignore the
number of available slots when deciding the number of processes to
launch.
#
[prte-rmaps-base:not-all-mapped-alloc]
Some of the requested hosts are not included in the current allocation for the
application:
%s
The requested hosts were:
%s
Verify that you have mapped the allocated resources properly using the
--host or --hostfile specification.
[prte-rmaps-base:no-mapped-node]
There are no allocated resources for the application:
%s
that match the requested mapping:
%s: %s
Verify that you have mapped the allocated resources properly for the
indicated specification.
[prte-rmaps-base:nolocal-no-available-resources]
There are no available nodes allocated to this job. This could be because
no nodes were found or all the available nodes were already used.
Note that since the -nolocal option was given no processes can be
launched on the local node.
#
[prte-rmaps-base:no-available-resources]
No nodes are available for this job, either due to a failure to
allocate nodes to the job, or allocated nodes being marked
as unavailable (e.g., down, rebooting, or a process attempting
to be relocated to another node when none are available).
[prte-rmaps-base:all-available-resources-used]
All nodes which are allocated for this job are already filled.
#
[out-of-vpids]
The system has exhausted its available ranks - the application is attempting
to spawn too many daemons and will be aborted.
This may be resolved by increasing the number of available ranks by
re-configuring with the --enable-jumbo-apps option, and then
re-building the application.
#
[rmaps:too-many-procs]
Your job has requested a conflicting number of processes for the
application:
App: %s
number of procs: %d
This is more processes than we can launch under the following
additional directives and conditions:
%s: %d
%s: %d
Please revise the conflict and try again.
#
[too-many-cpus-per-rank]
Your job has requested more cpus per process(rank) than there
are cpus in a package:
Cpus/rank: %d
#cpus/package: %d
Please correct one or both of these values and try again.
#
[failed-map]
Your job failed to map. Either no mapper was available, or none
of the available mappers was able to perform the requested
mapping operation.
Mapper result: %s
Application: %s
#procs to be mapped: %d
Mapping policy: %s
Binding policy: %s
#
[span-packages-multiple]
Your job failed to map because the resulting process placement
would cause the process to be bound to CPUs in more than one
package:
Mapping policy: %s
Binding policy: %s
CPUs/rank: %d
This configuration almost always results in a loss of performance
that can significantly impact applications. Either alter the
mapping, binding, and/or cpus/rank policies so that each process
can fit into a single package, or consider using an alternative
mapper that can handle this configuration (e.g., the rankfile mapper).
#
[span-packages-cpuset]
Your job failed to map because the resulting process placement
would cause the process to be bound to CPUs in more than one
package:
Mapping policy: %s
Binding policy: %s
PE-LIST: %s
This configuration almost always results in a loss of performance
that can significantly impact applications. Either alter the
mapping, binding, and/or PE-LIST policies so that each process
can fit into a single package, or consider using an alternative
mapper that can handle this configuration (e.g., the rankfile mapper).
#
[unrecognized-policy]
The specified %s directive is not recognized:
Directive: %s
Please check for a typo or ensure that the directive is a supported
one.
#
[oversubscribe-conflict]
Conflicting directives for the default oversubscription policy were
given stating that oversubscription was both allowed and not allowed.
Please check that only one policy is defined.
#
[rmaps:binding-target-not-found]
A request was made to bind to %s, but an appropriate target could not
be found on node %s.
#
[allocation-overload]
A request was made to bind that would require binding
processes to more cpus than are available in your allocation:
Application: %s
#processes: %d
Mapping policy: %s
Binding policy: %s
You can override this protection by adding the "overload-allowed"
option to your binding directive.
#
[rmaps:no-topology]
A mapping directive was given that requires knowledge of
a remote node's topology. However, no topology info is
available for the following node:
Node: %s
The job cannot be executed under this condition. Please either
remove the directive or investigate the lack of topology info.
#
[rmaps:no-available-cpus]
While computing bindings, we found no available cpus on
the following node:
Node: %s
Please check your allocation.
#
[rmaps:cpubind-not-supported]
A request was made to bind a process, but at least one node does NOT
support binding processes to cpus.
Node: %s
PRRTE uses the "hwloc" library to perform process and memory
binding. This error message means that hwloc has indicated that
processor binding support is not available on this machine.
On OS X, processor and memory binding is not available at all (i.e.,
the OS does not expose this functionality).
On Linux, lack of the functionality can mean that you are on a
platform where processor and memory affinity is not supported in Linux
itself, or that hwloc was built without NUMA and/or processor affinity
support. When building hwloc (which, depending on your PRRTE
installation, may be embedded in PRRTE itself), it is important to
have the libnuma header and library files available. Different linux
distributions package these files under different names; look for
packages with the word "numa" in them. You may also need a developer
version of the package (e.g., with "dev" or "devel" in the name) to
obtain the relevant header files.
If you are getting this message on a non-OS X, non-Linux platform,
then hwloc does not support processor / memory affinity on this
platform. If the OS/platform does actually support processor / memory
affinity, then you should contact the hwloc maintainers:
https://github.com/open-mpi/hwloc.
#
[rmaps:membind-not-supported]
WARNING: a request was made to bind a process. While the system
supports binding the process itself, at least one node does NOT
support binding memory to the process location.
Node: %s
PRRTE uses the "hwloc" library to perform process and memory
binding. This error message means that hwloc has indicated that
processor binding support is not available on this machine.
On OS X, processor and memory binding is not available at all (i.e.,
the OS does not expose this functionality).
On Linux, lack of the functionality can mean that you are on a
platform where processor and memory affinity is not supported in Linux
itself, or that hwloc was built without NUMA and/or processor affinity
support. When building hwloc (which, depending on your PRRTE
installation, may be embedded in PRRTE itself), it is important to
have the libnuma header and library files available. Different linux
distributions package these files under different names; look for
packages with the word "numa" in them. You may also need a developer
version of the package (e.g., with "dev" or "devel" in the name) to
obtain the relevant header files.
If you are getting this message on a non-OS X, non-Linux platform,
then hwloc does not support processor / memory affinity on this
platform. If the OS/platform does actually support processor / memory
affinity, then you should contact the hwloc maintainers:
https://github.com/open-mpi/hwloc.
This is a warning only; your job will continue, though performance may
be degraded.
#
[rmaps:membind-not-supported-fatal]
A request was made to bind a process. While the system
supports binding the process itself, at least one node does NOT
support binding memory to the process location.
Node: %s
PRRTE uses the "hwloc" library to perform process and memory
binding. This error message means that hwloc has indicated that
processor binding support is not available on this machine.
On OS X, processor and memory binding is not available at all (i.e.,
the OS does not expose this functionality).
On Linux, lack of the functionality can mean that you are on a
platform where processor and memory affinity is not supported in Linux
itself, or that hwloc was built without NUMA and/or processor affinity
support. When building hwloc (which, depending on your PRRTE
installation, may be embedded in PRRTE itself), it is important to
have the libnuma header and library files available. Different linux
distributions package these files under different names; look for
packages with the word "numa" in them. You may also need a developer
version of the package (e.g., with "dev" or "devel" in the name) to
obtain the relevant header files.
If you are getting this message on a non-OS X, non-Linux platform,
then hwloc does not support processor / memory affinity on this
platform. If the OS/platform does actually support processor / memory
affinity, then you should contact the hwloc maintainers:
https://github.com/open-mpi/hwloc.
The provided memory binding policy requires that PRRTE abort the
job at this time.
#
[rmaps:no-bindable-objects]
No bindable objects of the specified type were available
on at least one node:
Node: %s
Target: %s
#
[rmaps:unknown-binding-level]
Unknown binding level:
Target: %s
Cache level: %u
#
[prte-rmaps-base:missing-daemon]
While attempting to build a map of this job, a node
was detected to be missing a daemon:
Node: %s
This usually indicates a mismatch between what the
allocation provided for the node name versus what was
actually found on the node.
#
[prte-rmaps-base:no-objects]
No objects of the specified type were found on at least one node:
Type: %s
Node: %s
The map cannot be done as specified.
#
[topo-file]
A topology file was given for the compute nodes, but
we were unable to correctly process it. Common errors
include incorrectly specifying the path to the file,
or the file being generated in a way that is incompatible
with the version of hwloc being used by OMPI.
File: %s
Please correct the problem and try again.
#
[deprecated]
The following command line options and corresponding MCA parameter have
been deprecated and replaced as follows:
Command line options:
Deprecated: %s
Replacement: %s
Equivalent MCA parameter:
Deprecated: %s
Replacement: %s
The deprecated forms *will* disappear in a future version of PRRTE.
Please update to the new syntax.
#
[mismatch-binding]
A request for multiple cpus-per-proc was given, but a conflicting binding
policy was specified:
#cpus-per-proc: %d
type of cpus: %s
binding policy given: %s
The correct binding policy for the given type of cpu is:
correct binding policy: %s
This is the binding policy we would apply by default for this
situation, so no binding need be specified. Please correct the
situation and try again.
#
[mapping-too-low]
A request for multiple cpus-per-proc was given, but a directive
was also given to map to an object level that has less cpus than
requested ones:
#cpus-per-proc: %d
number of cpus: %d
map-by: %s
Please specify a mapping level that has more cpus, or else let us
define a default mapping that will allow multiple cpus-per-proc.
#
[unrecognized-modifier]
The mapping request contains an unrecognized qualifier:
Request: %s
Please check your request and try again. Remember, multiple
qualifiers must be separated by a colon (':').
#
[invalid-pattern]
The mapping request contains a pattern that doesn't match
the required syntax of #:object
Pattern: %s
Please check your request and try again.
#
[prte-rmaps-base:oversubscribed]
The requested number of processes exceeds the allocated
number of slots:
#slots: %d
#processes: %d
This creates an oversubscribed condition that may adversely
impact performance when combined with the requested binding
operation. We will continue, but will not bind the processes.
This warning can be omitted by adding the "overload-allowed"
qualifier to the binding policy.
#
[cannot-launch]
Although we were able to map your job, we are unable to launch
it at this time due to required resources being busy. Please
try again later.
#
[rmaps:no-locale]
The request to bind processes could not be completed due to
an internal error - the locale of the following process was
not set by the mapper code:
Process: %s
Please contact the OMPI developers for assistance. Meantime,
you will still be able to run your application without binding
by specifying "--bind-to none" on your command line.
#
[mapping-too-low-init]
A request for multiple cpus-per-proc was given, but a directive
was also given to map to an object level that cannot support that
directive.
Please specify a mapping level that has more than one cpu, or
else let us define a default mapping that will allow multiple
cpus-per-proc.
#
[device-not-specified]
The request to map processes by distance could not be completed
because device to map near by was not specified. Please, use
rmaps_dist_device mca parameter to set it.
#
[num-procs-not-specified]
Either the -host or -hostfile options were given, but the number
of processes to start was omitted. This combination is not supported.
Please specify the number of processes to run and try again.
#
[failed-assignments]
The attempt to assign hardware locations to processes on a
compute node failed:
Node: %s
Policy: %s
We cannot continue - please check that the policy is in
accordance with the actual available hardware.
#
[rmaps:insufficient-cpus]
The request to bind processes to cpus in a provided list
of logical id's based on their local rank on a node cannot
be met due to there being more processes on a node than
available cpus:
Node: %s
Local rank: %d
Cpu list: %s
Please adjust either the number of processes per node or
the list of cpus.
#
[unsupported-default-modifier]
A %s modifier was provided that is not supported as a default value:
Modifier: %s
You can provide this modifier on a per-job basis, but it cannot
be the default setting.
#
[unsupported-default-policy]
A %s policy was provided that is not supported as a default value:
Policy: %s
You can provide this policy on a per-job basis, but it cannot
be the default setting.
#
[missing-value]
A %s was given that requires a value, but no value for it was given
or the value is missing the required '=' sign:
Policy: %s
Given: %s
Please provide a value for this policy, or remove it.
#
[insufficient-cpus]
An insufficient number of CPUs was found while attempting to map the job
using the given request:
Mapping policy: %s
Default cpuset: %s
CPU list: %s
Available CPUs: %s
Please modify either the external limit set by the resource manager (e.g.,
cgroup), the default cpuset specified, or the cpu list provided as the
mapping policy modifier.
#
[insufficient-cpus-per-proc]
A request was made to bind a given number of cpus to each process, but
not enough cpus are available to meet the request:
Binding policy: %s
Node: %s
CPU envelope: %s
CPUs per proc: %d
Please adjust your request and try again.
[invalid-value]
A %s was given that requires a value, but the provided value is invalid:
Policy: %s
Given: %s
Please provide a valid value for this policy, or remove it.
#
[rankfile-no-filename]
The request to map processes using a rankfile could not be completed
because the filename of the rankfile was not specified. Please
specify the name of the rankfile.
#
[missing-modifier]
A ':' was found in a modifier specification but there is no modifier
following the ':'. Please specify a modifier.
#
[invalid-combination]
We cannot bind to hardware threads if we are not treating hardware
threads as independent CPUs since the lowest allocatable unit is
"core" by default. In order to bind to hardware threads, you must
therefore specify that we treat hardware threads as independent CPUs
by adding the "HWTCPUS" modifier to the "--map-by" directive.
The combination "--map-by core:hwtcpus --bind-to hwthread" or
"--map-by :hwtcpus --bind-to hwthread" will result in one process
being placed on each core, with the process bound to the first
hardware thread in that core.
In contrast, a case such as "--map-by package:hwtcpus --bind-to hwthread"
would result in each process being bound to successive hardware threads
on each package.
#
[unrecognized-qualifier]
The %s directive contains an unrecognized qualifier:
Qualifier: %s
Valid qualifiers: %s
Please check for a typo or ensure that the qualifier is a supported one.
#
[unrecognized-directive]
The specified %s directive is not recognized:
Directive: %s
Valid directives: %s
Please check for a typo or ensure that the directive is a supported one.
#
[missing-qualifier]
The %s option contains a directive that is missing a value:
Directive: %s
Valid directive: "%s=<value>"
Please check for a typo or ensure that the value is provided.
#
[missing-personality]
PRRTE has hit an internal problem that should never happen - it is
attempting to map a job that lacks an assigned personality.
Job: %s
Please report this to the PRRTE developers (https://github.com/openpmix/prrte/issues)
#
[unsupported-combination]
A %s policy was provided that is not supported in combination
with the "PE=N" option:
Policy: %s
When specifying the number of CPUs to use for each process in a job,
the processes must be bound at the CPU level - either HWThread if
"HWTCPUS" was specified, or CORE. Please remove the bind policy
specification and try again.
#
[unsupported-mapping-combo]
A mapping policy was provided that is not supported in combination
with the "PE-LIST=x,y,z" option:
Policy: %s
When specifying the CPUs to be used for the job, the mapper can
only look at those CPUs. It is therefore not possible to map
the job according to some other target object. Please either
remove the mapping policy or change it to "slot" and try again.
[must-map-by-obj]
When ranking by FILL or by SPAN, you must map by an object
type (e.g., HWTHREAD, NUMA, or cache level):
Map policy: %s
Rank policy: %s
Please specify a supported combination and try again.
#
[out-of-resource]
Either there are not enough slots available in the system to launch
the %d processes that were requested by the application, or there are
not enough CPUs to bind them as requested:
App: %s
Mapping: %s
Binding: %s
Either request fewer processes for your application, make more slots
available for use by expanding the allocation, or do not bind the
processes so that the number of CPUs is no longer a limiting factor.
A "slot" is the PRRTE term for an allocatable unit where we can
launch a process. The number of slots available are defined by the
environment in which PRRTE processes are run:
1. Hostfile, via "slots=N" clauses (N defaults to number of
processor cores if not provided)
2. The --host command line parameter, via a ":N" suffix on the
hostname (N defaults to 1 if not provided)
3. Resource manager (e.g., SLURM, PBS/Torque, LSF, etc.)
4. If none of a hostfile, the --host command line parameter, or an
RM is present, PRRTE defaults to the number of processor cores
In all the above cases, if you want PRRTE to default to the number
of hardware threads instead of the number of processor cores, use the
--use-hwthread-cpus option.
Alternatively, you can use the --map-by :OVERSUBSCRIBE option to ignore the
number of available slots when deciding the number of processes to
launch. Similarly, you can use the --bind-to :OVERLOAD option to bind
more than one process to a CPU, if desired, or --bind-to NONE to avoid
binding altogether.
#
# DEPRECATED OPTIONS
#
[placement-deprecated]
------------------------------ Deprecated Options ----------------------------
These deprecated options will be removed in a future release.
`--bind-to-core`
: **(Deprecated: Use `--bind-to core`)**
Bind processes to cores
`-bind-to-socket, --bind-to-socket`
: **(Deprecated: Use `--bind-to package`)**
Bind processes to processor sockets
`--bycore`
: **(Deprecated: Use `--map-by core`)**
Map processes by core
`-bynode, --bynode`
: **(Deprecated: Use `--map-by node`)**
Launch processes one per node, cycling by node in a round-robin
fashion. This spreads processes evenly among nodes and assigns ranks
in a round-robin, "by node" manner.
`--byslot`
: **(Deprecated: Use `--map-by slot`)**
Map and rank processes round-robin by slot.
`--cpus-per-proc <#perproc>`
: **(Deprecated: Use `--map-by <obj>:PE=<#perproc>`)**
Bind each process to the specified number of cpus.
`--cpus-per-rank <#perrank>`
: **(Deprecated: Use `--map-by <obj>:PE=<#perrank>`)**
Alias for `--cpus-per-proc`.
`--display-allocation`
: **(Deprecated: Use `--display ALLOC`)**
Display the detected resource allocation.
`--display-devel-map`
: **(Deprecated: Use `--display MAP-DEVEL`)**
Display a detailed process map (mostly intended for developers) just
before launch.
`--display-map`
: **(Deprecated: Use `--display MAP`)**
Display a table showing the mapped location of each process prior to
launch.
`--display-topo`
: **(Deprecated: Use `--display TOPO`)**
Display the topology as part of the process map (mostly intended for
developers) just before launch.
`--do-not-launch`
: **(Deprecated: Use `--map-by :DONOTLAUNCH`)**
Perform all necessary operations to prepare to launch the application,
but do not actually launch it (usually used to test mapping patterns).
`--do-not-resolve`
: **(Deprecated: Use `--map-by :DONOTRESOLVE`)**
Do not attempt to resolve interfaces - usually used to determine proposed
process placement/binding prior to obtaining an allocation.
`-N <num>`
: **(Deprecated: Use `--map-by prr:<num>:node`)**
Launch `num` processes per node on all allocated nodes.
`--nolocal`
: **(Deprecated: Use `--map-by :NOLOCAL`)**
Do not run any copies of the launched application on the same node
as `prun` is running. This option will override listing the `localhost`
with `--host` or any other host-specifying mechanism.
`--nooversubscribe`
: **(Deprecated: Use `--map-by :NOOVERSUBSCRIBE`)**
Do not oversubscribe any nodes; error (without starting any
processes) if the requested number of processes would cause
oversubscription. This option implicitly sets "max_slots" equal
to the "slots" value for each node. (Enabled by default).
`--npernode <#pernode>`
: **(Deprecated: Use `--map-by ppr:<#pernode>:node`)**
On each node, launch this many processes.
`--npersocket <#persocket>`
: **(Deprecated: Use `--map-by ppr:<#perpackage>:package`)**
On each node, launch this many processes times the number of
processor sockets on the node. The `--npersocket` option also turns
on the `--bind-to socket` option. The term `socket` has been globally
replaced with `package`.
`--oversubscribe`
: **(Deprecated: Use `--map-by :OVERSUBSCRIBE`)**
Nodes are allowed to be oversubscribed, even on a managed system,
and overloading of processing elements.
`--pernode`
: **(Deprecated: Use `--map-by ppr:1:node`)**
On each node, launch one process.
`--ppr`
: **(Deprecated: Use `--map-by ppr:<list>`)**
Comma-separated list of number of processes on a given resource type
[default: none].
`--rankfile <FILENAME>`
: **(Deprecated: Use `--map-by rankfile:FILE=<FILENAME>`)**
Use a rankfile for mapping/ranking/binding
`--report-bindings`
: **(Deprecated: Use `--display BINDINGS`)**
Report any bindings for launched processes.
`--tag-output`
: **(Deprecated: Use `--output TAG`)**
Tag all output with [job,rank]
`--timestamp-output`
: **(Deprecated: Use `--output TIMESTAMP`)**
Timestamp all application process output
`--use-hwthread-cpus`
: **(Deprecated: Use `--map-by :HWTCPUS`)**
Use hardware threads as independent cpus.
`--xml`
: **(Deprecated: Use `--output XML`)**
Provide all output in XML format
#
# ALL
#
[placement-all]
#include#help-schizo-detail#placement
#include#help-schizo-detail#placement-examples
#include#help-schizo-detail#placement-fundamentals
#include#help-schizo-detail#placement-limits
#include#help-schizo-detail#placement-diagnostics
#include#help-schizo-detail#placement-rankfiles

71
macx64/mpi/openmpi/share/prte/help-prte-rmaps-ppr.txt Normal file
View File

@@ -0,0 +1,71 @@
# -*- text -*-
#
# Copyright (c) 2011 Cisco Systems, Inc. All rights reserved.
# Copyright (c) 2019-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
#
[invalid-ppr]
An invalid value was given for the number of processes
per resource (ppr) to be mapped on each node:
PPR: %s
The specification must be a comma-separated list containing
combinations of number, followed by a colon, followed
by the resource type. For example, a value of "1:package" indicates that
one process is to be mapped onto each package. Values are supported
for "hwthread", "core", "package",
"l1cache", "l2cache", "l3cache", and "node" . Note that
enough characters must be provided to clearly specify the desired
resource (e.g., "l1" for "l1cache").
#
[unrecognized-ppr-option]
An unrecognized value was given for the number of processes
per resource (ppr) to be mapped on each node:
Value: %s
PPR: %s
The specification must be a number, followed by a colon, followed
by the resource type. For example, a value of "1:slot" indicates that
anything over one process per slot is to be considered oversubscribed.
Only values for "hwthread", "core", "package",
"l1cache", "l2cache", "l3cache", and "node" are allowed. Note that
enough characters must be provided to clearly specify the desired
resource (e.g., "l1" for "l1cache").
#
[ppr-violation]
The provided mapping directives resulted in too many processes
being placed on a node:
Node: %s
Num procs: %d
Limiting resource: %s
Num resources: %d
Specified constraint: %s
Please adjust and try again.
#
[ppr-too-many-procs]
Your job has requested more processes than the ppr for
this topology can support:
App: %s
Number of procs: %d
Procs mapped: %d
Total number of procs: %d
PPR: %s
Please revise the conflict and try again.
#
[ppr-topo-missing]
A ppr pattern was specified, but the topology information
for the following node is missing:
Node: %s

91
macx64/mpi/openmpi/share/prte/help-prte-rmaps-rr.txt Normal file
View File

@@ -0,0 +1,91 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2017-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2018-2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2022 Nanook Consulting. All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English general help file for PRTE's prun.
#
[prte-rmaps-rr:multi-apps-and-zero-np]
RMAPS found multiple applications to be launched, with
at least one that failed to specify the number of processes to execute.
When specifying multiple applications, you must specify how many processes
of each to launch via the -np argument.
#
[prte-rmaps-rr:per-node-and-too-many-procs]
There are not enough nodes in your allocation to satisfy your request to launch
%d processes on a per-node basis - only %d nodes were available.
Either request fewer processes, or obtain a larger allocation.
#
[prte-rmaps-rr:n-per-node-and-too-many-procs]
There are not enough nodes in your allocation to satisfy your request to launch
%d processes on a %d per-node basis - only %d nodes with a total of %d slots were available.
Either request fewer processes, or obtain a larger allocation.
[prte-rmaps-rr:n-per-node-and-not-enough-slots]
There are not enough slots on the nodes in your allocation to satisfy your request to launch on a %d process-per-node basis - only %d slots/node were available.
Either request fewer processes/node, or obtain a larger allocation.
#
[prte-rmaps-rr:no-np-and-user-map]
You have specified a rank-to-node/slot mapping, but failed to provide
the number of processes to be executed. For some reason, this information
could not be obtained from the mapping you provided, so we cannot continue
with executing the specified application.
#
[prte-rmaps-rr:not-enough-objs]
There are not enough resources on the available nodes
to meet the requested mapping.
Application: %s
Number of procs: %d
Number of resources: %d
#
[prte-rmaps-rr:not-enough-cpus]
There are not enough slots available in the system or not enough
CPUs in the specified PE-LIST to map the number of processes requested
by the application:
app: %s
Number of procs: %d
pe-list: %s
Either request fewer procs for your application, make more slots
available for use, or expand the pe-list.
A "slot" is the PRRTE term for an allocatable unit where we can
launch a process. The number of slots available are defined by the
environment in which PRRTE processes are run:
1. Hostfile, via "slots=N" clauses (N defaults to number of
processor cores if not provided)
2. The --host command line parameter, via a ":N" suffix on the
hostname (N defaults to 1 if not provided)
3. Resource manager (e.g., SLURM, PBS/Torque, LSF, etc.)
4. If none of a hostfile, the --host command line parameter, or an
RM is present, PRRTE defaults to the number of processor cores
In all the above cases, if you want PRRTE to default to the number
of hardware threads instead of the number of processor cores, use the
--use-hwthread-cpus option.
Alternatively, you can use the --map-by :OVERSUBSCRIBE option to ignore
the number of available slots and size of the pe-list when placing the
processes.
#

78
macx64/mpi/openmpi/share/prte/help-prte-rmaps-seq.txt Normal file
View File

@@ -0,0 +1,78 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2017-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2018-2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2021-2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English general help file for PRTE's prun.
#
[prte-rmaps-seq:resource-not-found]
The specified sequence file contained a node (%s) that is not in your
allocation. We therefore cannot map a process rank to it. Please
check your allocation and sequence file to ensure the latter only
contains allocated nodes.
#
[seq:not-enough-resources]
A sequential map was requested, but not enough node entries
were given to support the requested number of processes:
Num procs: %d
Num nodes: %d
The sequential mapper requires that there be a node entry for
every process in the job, and it processes the provided file
using each node entry to identify the node where that numbered
rank is to be placed.
Note that you can provide a separate sequence file (different
from a hostfile that specifies the number of slots on each node)
for use by the sequential mapper in its assignments by using the
appropriate modifier:
Example: --map-by seq:file=myseqfile
#
[missing-cpu]
While parsing the sequential hostfile, %s encountered a line that
specified a non-existent CPU:
Slots: %s
Available CPUs: %s
Please correct the line and try again.
#
[bad-syntax]
Error, invalid syntax in sequential hostfile %s.
The sequential mapper requires that there be a node entry for
every process in the job, and it processes the provided file
using each node entry to identify the node where that numbered
rank is to be placed. You can specify the CPU binding for the
process on its node entry line in the following manner:
A 1:0,1 # put this rank on node A, bound to socket 1, CPUs 0 and 1
B 1-4 # put this rank on node B, bound to CPUs 1-4
Please correct the syntax and try again.
#
[proc-failed-to-map]
A process could not be mapped:
Hostname: %s
App: %s
Could not continue.

287
macx64/mpi/openmpi/share/prte/help-prte-rtc-base.txt Normal file
View File

@@ -0,0 +1,287 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2011-2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2011 Los Alamos National Security, LLC.
# All rights reserved.
# Copyright (c) 2014-2020 Intel, Inc. All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English general help file for PRTE's prun.
#
[prte-rtc-base:not-all-mapped-alloc]
Some of the requested hosts are not included in the current allocation for the
application:
%s
The requested hosts were:
%s
Verify that you have mapped the allocated resources properly using the
--host or --hostfile specification.
[prte-rtc-base:no-mapped-node]
There are no allocated resources for the application:
%s
that match the requested mapping:
%s: %s
Verify that you have mapped the allocated resources properly for the
indicated specification.
[prte-rtc-base:nolocal-no-available-resources]
There are no available nodes allocated to this job. This could be because
no nodes were found or all the available nodes were already used.
Note that since the -nolocal option was given no processes can be
launched on the local node.
[prte-rtc-base:no-available-resources]
No nodes are available for this job, either due to a failure to
allocate nodes to the job, or allocated nodes being marked
as unavailable (e.g., down, rebooting, or a process attempting
to be relocated to another node when none are available).
[prte-rtc-base:all-available-resources-used]
All nodes which are allocated for this job are already filled.
#
[out-of-vpids]
The system has exhausted its available ranks - the application is attempting
to spawn too many daemons and will be aborted.
This may be resolved by increasing the number of available ranks by
re-configuring with the --enable-jumbo-apps option, and then
re-building the application.
#
[rtc:too-many-procs]
Your job has requested a conflicting number of processes for the
application:
App: %s
number of procs: %d
This is more processes than we can launch under the following
additional directives and conditions:
%s: %d
%s: %d
Please revise the conflict and try again.
#
[too-many-cpus-per-rank]
Your job has requested more cpus per process(rank) than there
are cpus in a package:
Cpus/rank: %d
#cpus/package: %d
Please correct one or both of these values and try again.
#
[failed-map]
Your job failed to map. Either no mapper was available, or none
of the available mappers was able to perform the requested
mapping operation. This can happen if you request a map type
(e.g., loadbalance) and the corresponding mapper was not built.
#
[unrecognized-policy]
The specified %s policy is not recognized:
Policy: %s
Please check for a typo or ensure that the option is a supported
one.
#
[redefining-policy]
Conflicting directives for %s policy are causing the policy
to be redefined:
New policy: %s
Prior policy: %s
Please check that only one policy is defined.
#
[rtc:binding-target-not-found]
A request was made to bind to %s, but an appropriate target could not
be found on node %s.
#
[rtc:binding-overload]
A request was made to bind to that would result in binding more
processes than cpus on a resource:
Bind to: %s
Node: %s
#processes: %d
#cpus: %d
You can override this protection by adding the "overload-allowed"
option to your binding directive.
#
[rtc:no-topology]
A mapping directive was given that requires knowledge of
a remote node's topology. However, no topology info is
available for the following node:
Node: %s
The job cannot be executed under this condition. Please either
remove the directive or investigate the lack of topology info.
#
[rtc:no-available-cpus]
While computing bindings, we found no available cpus on
the following node:
Node: %s
Please check your allocation.
#
[rtc:cpubind-not-supported]
A request was made to bind a process, but at least one node does NOT
support binding processes to cpus.
Node: %s
#
[rtc:membind-not-supported]
WARNING: a request was made to bind a process. While the system
supports binding the process itself, at least one node does NOT
support binding memory to the process location.
Node: %s
This is a warning only; your job will continue, though performance may
be degraded.
#
[rtc:membind-not-supported-fatal]
A request was made to bind a process. While the system
supports binding the process itself, at least one node does NOT
support binding memory to the process location.
Node: %s
The provided memory binding policy requires that we abort the
job at this time.
#
[rtc:no-bindable-objects]
No bindable objects of the specified type were available
on at least one node:
Node: %s
Target: %s
#
[rtc:unknown-binding-level]
Unknown binding level:
Target: %s
Cache level: %u
#
[prte-rtc-base:missing-daemon]
While attempting to build a map of this job, a node
was detected to be missing a daemon:
Node: %s
This usually indicates a mismatch between what the
allocation provided for the node name versus what was
actually found on the node.
#
[prte-rtc-base:no-objects]
No objects of the specified type were found on at least one node:
Type: %s
Node: %s
The map cannot be done as specified.
#
[topo-file]
A topology file was given for the compute nodes, but
we were unable to correctly process it. Common errors
include incorrectly specifying the path to the file,
or the file being generated in a way that is incompatible
with the version of hwloc being used by OMPI.
File: %s
Please correct the problem and try again.
#
[deprecated]
The following command line options and corresponding MCA parameter have
been deprecated and replaced as follows:
Command line options:
Deprecated: %s
Replacement: %s
Equivalent MCA parameter:
Deprecated: %s
Replacement: %s
The deprecated forms *will* disappear in a future version of PRTE.
Please update to the new syntax.
#
[mismatch-binding]
A request for multiple cpus-per-proc was given, but a conflicting binding
policy was specified:
#cpus-per-proc: %d
type of cpus: %s
binding policy given: %s
The correct binding policy for the given type of cpu is:
correct binding policy: %s
This is the binding policy we would apply by default for this
situation, so no binding need be specified. Please correct the
situation and try again.
#
[mapping-too-low]
A request for multiple cpus-per-proc was given, but a directive
was also given to map to an object level that has less cpus than
requested ones:
#cpus-per-proc: %d
number of cpus: %d
map-by: %s
Please specify a mapping level that has more cpus, or else let us
define a default mapping that will allow multiple cpus-per-proc.
#
[unrecognized-modifier]
The mapping request contains an unrecognized modifier:
Request: %s
Please check your request and try again.
#
[invalid-pattern]
The mapping request contains a pattern that doesn't match
the required syntax of #:object
Pattern: %s
Please check your request and try again.
#
[prte-rtc-base:oversubscribed]
The requested number of processes exceeds the allocated
number of slots:
#slots: %d
#processes: %d
This creates an oversubscribed condition that may adversely
impact performance when combined with the requested binding
operation. We will continue, but will not bind the processes.
This warning can be omitted by adding the "overload-allowed"
qualifier to the binding policy.
#
[cannot-launch]
Although we were able to map your job, we are unable to launch
it at this time due to required resources being busy. Please
try again later.

30
macx64/mpi/openmpi/share/prte/help-prte-rtc-hwloc.txt Normal file
View File

@@ -0,0 +1,30 @@
# -*- text -*-
#
# Copyright (c) 2009-2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2010-2012 Los Alamos National Security, LLC.
# All rights reserved.
#
# Copyright (c) 2017-2019 Intel, Inc. All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English help file for PRTE's hwloc support.
#
[sys call fail]
A system call failed during shared memory initialization that should
not have.
Local host: %s
System call: %s %s
Error: %s (errno %d)
#
[target full]
It appears as if there is not enough space for %s (the shared-memory backing
file for hwloc topology).
Local host: %s
Space Requested: %lu B
Space Available: %llu B

146
macx64/mpi/openmpi/share/prte/help-prte-runtime.txt Normal file
View File

@@ -0,0 +1,146 @@
[bogus section]
This section is not used by PRTE code. But we have to put a RST
section title in this file somewhere, or Sphinx gets unhappy. So we
put it in a section that is ignored by PRTE code.
Hello, world
============
[prte_init:startup:internal-failure]
It looks like "prte_init()" failed for some reason. There are many
reasons that can cause PRRTE to fail during "prte_init()", some of
which are due to configuration or environment problems. This failure
appears to be an internal failure — here's some additional information
(which may only be relevant to a PRRTE developer):
%s failed
--> Returned value %s (%d) instead of PRTE_SUCCESS
[prte:session:dir:prohibited]
The specified location for the temporary directories required by PRTE
is on the list of prohibited locations:
Location given: %s
Prohibited locations: %s
If you believe this is in error, please contact your system
administrator to have the list of prohibited locations changed.
Otherwise, please identify a different location to be used (use "-h"
to see the cmd line option), or simply let the system pick a default
location.
[prte:session:dir:nopwname]
PRTE was unable to obtain the username in order to create a path for
its required temporary directories. This type of error is usually
caused by a transient failure of network-based authentication services
(e.g., LDAP failure due to network congestion), but can also be an
indication of system misconfiguration.
Please consult your system administrator about these issues and try
again.
[prte_nidmap:too_many_nodes]
An error occurred while trying to pack the information about the job.
More nodes have been found than the %d expected. Please check your
configuration files such as the mapping.
[prte_init:startup:num_daemons]
PRTE was unable to determine the number of nodes in your allocation.
We are therefore assuming a very large number to ensure you receive
proper error messages.
[failed-to-uncompress]
A compressed message was received that could not be decompressed. This
is most likely due to a missing libz library on the receiving node:
node: %s
Please ensure that the "libz" library is present on all compute nodes.
[bootstrap-not-found]
Bootstrap startup was requested, but the required configuration file
was not found on at least one node:
node: %s
file: %s
Please ensure the file has been installed in the correct location on
every node included in this PRRTE DVM.
[bootstrap-missing-field-name]
Bootstrap startup was requested, but the required configuration file
on at least one node is missing the field name on an entry:
node: %s
file: %s
line: %s
Please correct the line.
[bootstrap-missing-value]
Bootstrap startup was requested, but the required configuration file
on at least one node is missing a value on an entry:
node: %s
file: %s
line: %s
Please correct the line.
[bootstrap-missing-entry]
Bootstrap startup was requested, but the required configuration file
on at least one node is missing a required entry:
node: %s
file: %s
entry: %s
Please provide the missing information.
[bootstrap-bad-nodelist]
Bootstrap startup was requested, but we were unable to parse the regex
of participating nodes on at least one node:
node: %s
file: %s
regex: %s
error: %s
Please fix the error and try again.
[prte:session:dir:shared]
PRTE has detected that the head of the session directory tree (where
scratch files and shared memory backing storage will be placed)
resides on a shared file system:
Directory: %s
File system type: %s
For performance reasons, it is strongly recommended that the session
directory be located on a local file system. This can be controlled by
setting the system temporary directory to be used by PRTE using either
the TMPDIR envar or the "prte_tmpdir_base" MCA param.
If you need the temporary directory to be different on remote nodes
from the local one where %s is running (e.g., when a login node is
being employed), then you can set the local temporary directory using
the "prte_local_tmpdir_base" MCA param and the one to be used on all
other nodes using the "prte_remote_tmpdir_base" param.
This is only a warning advisory and your job will continue. You can
disable this warning in the future by setting the
"prte_silence_shared_fs" MCA param to "1".

123
macx64/mpi/openmpi/share/prte/help-prte-util.txt Normal file
View File

@@ -0,0 +1,123 @@
[bogus section]
This section is not used by PRTE code. But we have to put a RST
section title in this file somewhere, or Sphinx gets unhappy. So we
put it in a section that is ignored by PRTE code.
Hello, world
============
[stacktrace signal override]
PRTE was inserting a signal handler for signal %d but noticed that
there is already a non-default handler installed. PRTE's handler was
therefore not installed; your job will continue. This warning message
will only be displayed once, even if PRTE encounters this situation
again.
To avoid displaying this warning message, you can either not install
the error handler for signal %d or you can have PRTE not try to
install its own signal handler for this signal by setting the
"prte_signals" MCA parameter.
Signal: %d
Current prte_signal value: %s
[stacktrace bad signal]
PRTE was inserting a signal handler but was given an invalid signal
number:
Signal string: %s
Bad value: %s
The given value must be an integer within the signal number range.
Please correct the value and try again.
[malformed net_private_ipv4]
PRTE has detected at least one malformed IP address or netmask in the
value of the prte_net_private_ipv4 MCA parameter. The
prte_net_private_ipv4 MCA parameter accepts a semicolon-delimited list
of Classless Inter-Domain Routing (CIDR) notation specifications, each
of the form <ipaddress>/<mask>. For example:
10.0.0.0/8;172.16.0.0/12;192.168.0.0/16;169.254.0.0/16
The first detected malformed entry was %s.
[invalid-net-mask]
PRTE has detected a malformed IPv4 address or netmask:
Value provided: %s
Accepted values follow the Classless Inter-Domain Routing (CIDR)
notation specifications. For example:
10.0.0.0/8
172.16/12
192.168
169.254.0.0/16
[malformed-uri]
PRTE has detected a malformed URI:
URI: %s
Accepted values follow IETF RFC3986, e.g.
"file://192.168.1.1/over/there"
[relative-path]
When creating a URI, all files must be specified in absolute paths:
Value provided: %s
Please update your application to provide the full path to the file.
[sys-limit-failed]
Per request, PRTE attempted to set a system resource limit to a given
value:
Resource: %s
Limit: %s
The system has refused to allow this operation. This is likely due to
a permission limitation, or specifying an unsupported value. Please
check the system or remove the request and try again.
[sys-limit-unrecognized]
PRTE received a request to set a system resource limit. Sadly, OMPI
does not recognize or currently support the specified resource:
Resource: %s
Limit: %s
Please correct the request and try again.
[dir-mode]
While working through a directory tree, we were unable to set a
directory to the desired mode:
Directory: %s
Mode: %0x
Error: %s
Please check to ensure you have adequate permissions to perform the
desired operation.
[mkdir-failed]
A call to mkdir was unable to create the desired directory:
Directory: %s
Error: %s
Please check to ensure you have adequate permissions to perform the
desired operation.

668
macx64/mpi/openmpi/share/prte/help-prte.txt Normal file
View File

@@ -0,0 +1,668 @@
[bogus section]
This section is not used by PRTE code. But we have to put a RST
section title in this file somewhere, or Sphinx gets unhappy. So we
put it in a section that is ignored by PRTE code.
Hello, world
============
[version]
%s (%s) %s
%s
[usage]
%s (%s) %s
Usage: %s [OPTION]...
Initiate an instance of the PMIx Reference RTE (PRRTE) DVM
The following list of command line options are available. Note that
more detailed help for any option can be obtained by adding that
option to the help request as "--help <option>".
+----------------------+-----------------------------------------------+
| | General Options |
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "-h" | "--help" | This help message |
+----------------------+-----------------------------------------------+
| "-h" | "--help | Help for the specified option |
| <arg0>" | |
+----------------------+-----------------------------------------------+
| "-v" | "--verbose" | Enable typical debug options |
+----------------------+-----------------------------------------------+
| "-V" | "--version" | Print version and exit |
+----------------------+-----------------------------------------------+
+----------------------+-----------------------------------------------+
| | Debug Options |
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "--debug-daemons" | Debug daemons - if not set, any "verbose" |
| | settings will be limited to the DVM |
| | controller to reduce clutter |
+----------------------+-----------------------------------------------+
| "--debug-daemons- | Enable debugging of any PRRTE daemons used by |
| file" | this application, storing their verbose |
| | output in files |
+----------------------+-----------------------------------------------+
| "--leave-session- | Do not discard stdout/stderr of remote PRRTE |
| attached" | daemons |
+----------------------+-----------------------------------------------+
| "--display <arg0>" | Comma-delimited list of options for |
| | displaying information |
+----------------------+-----------------------------------------------+
+----------------------+-----------------------------------------------+
| | DVM Options |
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "--runtime-options | Comma-delimited list of runtime directives |
| <options>" | for the job (e.g., show progress reports on |
| | DVM startup for large systems) |
+----------------------+-----------------------------------------------+
| "--default-hostfile | Provide a default hostfile |
| <filename>" | |
+----------------------+-----------------------------------------------+
| "-H|--host | List of hosts to use for the DVM |
| <hostname>" | |
+----------------------+-----------------------------------------------+
| "--hostfile | Provide a hostfile |
| <filename>" | |
+----------------------+-----------------------------------------------+
| "--machinefile | Provide a hostfile (synonym for "--hostfile") |
| <filename>" | |
+----------------------+-----------------------------------------------+
| "--pmixmca <key> | Pass PMIx MCA parameters |
| <value>" | |
+----------------------+-----------------------------------------------+
| "--prtemca <key> | Pass PRTE MCA parameters to the DVM |
| <value>" | |
+----------------------+-----------------------------------------------+
| "--show-progress" | Output a brief periodic report on DVM startup |
| | progress |
+----------------------+-----------------------------------------------+
| "-x | Export an environment variable, optionally |
| <var_name>[=value]" | specifying a value |
+----------------------+-----------------------------------------------+
| "--allow-run-as- | Allow execution as root (**STRONGLY |
| root" | DISCOURAGED**) |
+----------------------+-----------------------------------------------+
| "--daemonize" | Daemonize the DVM daemons into the background |
+----------------------+-----------------------------------------------+
| "--forward-signals | Comma-delimited list of additional signals |
| <signals>" | (names or integers) to forward |
+----------------------+-----------------------------------------------+
| "--keepalive <arg0>" | Pipe to monitor - DVM will terminate upon |
| | closure |
+----------------------+-----------------------------------------------+
| "--launch-agent | Name of daemon executable used to start |
| <executable>" | processes on remote nodes (default: "prted") |
+----------------------+-----------------------------------------------+
| "--max-vm-size | Max number of daemons to start |
| <size>" | |
+----------------------+-----------------------------------------------+
| "--no-ready-msg" | Do not print a "DVM ready" message |
+----------------------+-----------------------------------------------+
| "--noprefix" | Disable automatic "--prefix" behavior |
+----------------------+-----------------------------------------------+
| "--prefix <dir>" | Prefix to be used to look for RTE executables |
+----------------------+-----------------------------------------------+
| "--report-pid <arg>" | Print out PID on stdout ("-"), stderr ("+"), |
| | or a file (anything else) |
+----------------------+-----------------------------------------------+
| "--report-uri <arg>" | Print out URI on stdout ("-"), stderr ("+"), |
| | or a file (anything else) |
+----------------------+-----------------------------------------------+
| "--set-sid" | Direct the DVM daemons to separate from the |
| | current session |
+----------------------+-----------------------------------------------+
| "--singleton <id>" | ID of the singleton process that started us |
+----------------------+-----------------------------------------------+
| "--system-server" | Start the DVM as the system server |
+----------------------+-----------------------------------------------+
| "--tmpdir <dir>" | Set the filesystem root for the session |
| | directory tree |
+----------------------+-----------------------------------------------+
| "--tune <arg0>" | File(s) containing MCA params for tuning DVM |
| | operations |
+----------------------+-----------------------------------------------+
| "--timeout | Timeout DVM startup if time exceeds the |
| <seconds>" | specified number of seconds |
+----------------------+-----------------------------------------------+
Report bugs to %s
[prtemca]
Pass a PRRTE MCA parameter.
Syntax: "--prtemca <key> <value>", where "key" is the parameter name
and "value" is the parameter value.
[pmixmca]
Pass a PMIx MCA parameter
Syntax: "--pmixmca <key> <value>", where "key" is the parameter name
and "value" is the parameter value.
[tune]
Comma-delimited list of one or more files containing PRRTE and PMIx
MCA params for tuning DVM and/or application operations. Parameters in
the file will be treated as *generic* parameters and subject to the
translation rules/uncertainties. See "--help mca" for more
information.
Syntax in the file is:
param = value
with one parameter and its associated value per line. Empty lines and
lines beginning with the "#" character are ignored, as is any
whitespace around the "=" character.
[no-ready-msg]
Do not print a DVM ready message
[daemonize]
Daemonize the DVM daemons and controller into the background
[system-server]
Start the DVM controller and its daemons as the system server on their
nodes
[set-sid]
Direct the DVM (controller and daemons) to separate from the current
session
[report-pid]
Printout DVM controller's PID on stdout [-], stderr [+], or a file
[anything else]
[report-uri]
Printout DVM controller's URI on stdout [-], stderr [+], or a file
[anything else]
[test-suicide]
Test DVM cleanup upon daemon failure by having one daemon suicide
after delay
[default-hostfile]
Specify a default hostfile.
Also see "--help hostfile".
[singleton]
DVM is being started by a singleton process (i.e., one not started by
a DVM) - the argument must be the PMIx ID of the singleton process
that started us
[keepalive]
Pipe for DVM controller to monitor - DVM will terminate upon closure
[launch-agent]
Name of daemon executable used to start processes on remote nodes
(default: prted). This is the executable the DVM controller shall
start on each remote node when establishing the DVM.
[max-vm-size]
Maximum number of daemons to start - sets the maximum size of the DVM.
[debug-daemons]
Debug daemon output enabled. This is a somewhat limited stream of
information normally used to simply confirm that the daemons started.
Includes leaving the output streams open.
[debug-daemons-file]
Debug daemon output is enabled and all output from the daemons is
redirected into files with names of the form:
output-prted-<daemon-nspace>-<nodename>.log
These names avoid conflict on shared file systems. The files are
located in the top-level session directory assigned to the DVM.
See the "Session directory" HTML documentation for additional details
about the PRRTE session directory.
[leave-session-attached]
Do not discard stdout/stderr of remote PRRTE daemons. The primary use
for this option is to ensure that the daemon output streams (i.e.,
stdout and stderr) remain open after launch, thus allowing the user to
see any daemon-generated error messages. Otherwise, the daemon will
"daemonize" itself upon launch, thereby closing its output streams.
[tmpdir]
Define the root location for the PRRTE session directory tree
See the "Session directory" HTML documentation for additional details
about the PRRTE session directory.
[prefix]
Prefix to be used to look for PRRTE executables. PRRTE automatically
sets the prefix for remote daemons if it was either configured with
the "--enable-prte-prefix-by-default" option OR prte itself was
executed with an absolute path to the prte command. This option
overrides those settings, if present, and forces use of the provided
path.
[noprefix]
Disable automatic "--prefix" behavior. PRRTE automatically sets the
prefix for remote daemons if it was either configured with the "--
enable-prte-prefix-by-default" option OR prte itself was executed with
an absolute path to the "prte" command. This option disables that
behavior.
[forward-signals]
Comma-delimited list of additional signals (names or integers) to
forward to application processes ("none" = forward nothing). Signals
provided by default include SIGTSTP, SIGUSR1, SIGUSR2, SIGABRT,
SIGALRM, and SIGCONT.
[allow-run-as-root]
Allow execution as root **(STRONGLY DISCOURAGED)**.
Running as root exposes the user to potentially catastrophic file
system corruption and damage — e.g., if the user accidentally points
the root of the session directory to a system required point, this
directory and all underlying elements will be deleted upon job
completion, thereby rendering the system inoperable.
It is recognized that some environments (e.g., containers) may require
operation as root, and that the user accepts the risks in those
scenarios. Accordingly, one can override PRRTE's run-as-root
protection by providing one of the following:
* The "--allow-run-as-root" command line directive
* Adding **BOTH** of the following environmental parameters:
* "PRTE_ALLOW_RUN_AS_ROOT=1"
* "PRTE_ALLOW_RUN_AS_ROOT_CONFIRM=1"
Again, we recommend this only be done if absolutely necessary.
[no-aggregate-help]
Do not aggregate help output from multiple processes. PRRTE defaults
to aggregating messages generated by its "help" subsystem so that only
one is printed out per topic (along with the number of processes that
reported the same issue). This is done to avoid users receiving a
flood of one-per-process error messages, all containing the identical
error report. Setting this option turns off the aggregation, thereby
allowing the user to see duplicate reports from multiple processes.
[timeout]
Timeout DVM startup if time exceeds the specified number of seconds.
The DVM startup will abort after the specified interval.
[x]
Export an environment variable, optionally specifying a value. For
example:
* "-x foo" exports the environment variable "foo" and takes its value
from the current environment.
* "-x foo=bar" exports the environment variable name "foo" and sets
its value to "bar" in the started processes.
* "-x foo*" exports all current environmental variables starting with
"foo".
[show-progress]
Output a brief periodic report on DVM startup progress
[hostfile]
PRRTE supports several levels of user-specified host lists based on an
established precedence order. Users can specify a default hostfile
that contains a list of nodes to be used by the DVM. Only one default
hostfile can be provided for a given DVM. In addition, users can
specify a hostfile that contains a list of nodes to be used for a DVM,
or can provide a comma-delimited list of nodes to be used for that DVM
via the "--host" command line option.
The precedence order applied to these various options depends to some
extent on the local environment. The following table illustrates how
host and hostfile directives work together to define the set of hosts
upon which a DVM will execute in the absence of a resource manager
(RM):
+------------+---------+------------+----------------------------------+
| Default | host | hostfile | Result |
| hostfile | | | |
|============|=========|============|==================================|
| unset | unset | unset | The DVN will consist solely of |
| | | | the local host where the DVM was |
| | | | started. |
+------------+---------+------------+----------------------------------+
| unset | set | unset | Host option defines resource |
| | | | list for the DVM. |
+------------+---------+------------+----------------------------------+
| unset | unset | set | Hostfile option defines resource |
| | | | list for the DVM. |
+------------+---------+------------+----------------------------------+
| unset | set | set | Hostfile option defines resource |
| | | | list for the DVM, then host |
| | | | filters the list to define the |
| | | | final set of nodes to be used by |
| | | | the DVM |
+------------+---------+------------+----------------------------------+
| set | unset | unset | Default hostfile defines |
| | | | resource list for the DVM |
+------------+---------+------------+----------------------------------+
| set | set | unset | Default hostfile defines |
| | | | resource list for the DVM, then |
| | | | host filters the list to define |
| | | | the final set of nodes to be |
| | | | used by the DVM |
+------------+---------+------------+----------------------------------+
| set | set | set | Default hostfile defines |
| | | | resource list for the DVM, then |
| | | | hostfile filters the list, and |
| | | | then host filters the list to |
| | | | define the final set of nodes to |
| | | | be used by the DVM |
+------------+---------+------------+----------------------------------+
This changes somewhat in the presence of an RM as that entity
specifies the initial allocation of nodes. In this case, the default
hostfile, hostfile and host directives are all used to filter the RM's
specification so that a user can utilize different portions of the
allocation for different DVMs. This is done according to the same
precedence order as in the prior table, with the RM providing the
initial pool of nodes.
Hostfiles (sometimes called "machine files") are a combination of two
things:
1. A listing of hosts on which to launch processes.
2. Optionally, limit the number of processes which can be launched on
each host.
Hostfile syntax consists of one node name on each line, optionally
including a designated number of "slots":
# This is a comment line, and will be ignored
node01 slots=10
node13 slots=5
node15
node16
node17 slots=3
...
Blank lines and lines beginning with a "#" are ignored.
A "slot" is the PRRTE term for an allocatable unit where we can launch
a process. See the section on definition of the term "slot" for a
longer description of slots.
In the absence of the "slot" parameter, PRRTE will assign either the
number of slots to be the number of CPUs detected on the node or the
resource manager-assigned value if operating in the presence of an RM.
Important:
If using a resource manager, the user-specified number of slots is
capped by the RM-assigned value.
See the "Host specification" HTML documentation for details about the
format and content of hostfiles.
[machinefile]
Provide a hostfile. This option is a synonym for "--hostfile"; see
that option for more information.
[host]
Host syntax consists of a comma-delimited list of node names, each
entry optionally containing a ":N" extension indicating the number of
slots to assign to that entry:
--host node01:5,node02
In the absence of the slot extension, one slot will be assigned to the
node. Duplicate entries are aggregated and the number of slots
assigned to that node are summed together.
Note:
A "slot" is the PRRTE term for an allocatable unit where we can
launch a process. Thus, the number of slots equates to the maximum
number of processes PRRTE may start on that node without
oversubscribing it.
[display]
The "display" command line directive must be accompanied by a comma-
delimited list of case-insensitive options indicating what information
about the job and/or allocation is to be displayed. The full directive
need not be provided — only enough characters are required to uniquely
identify the directive. For example, "ALL" is sufficient to represent
the "ALLOCATION" directive — while "MAP" can not be used to represent
"MAP-DEVEL" (though "MAP-D" would suffice).
Supported values include:
* "ALLOCATION" displays the detected hosts and slot assignments for
this job
* "BINDINGS" displays the resulting bindings applied to processes in
this job
* "MAP" displays the resulting locations assigned to processes in this
job
* "MAP-DEVEL" displays a more detailed report on the locations
assigned to processes in this job that includes local and node
ranks, assigned bindings, and other data
* "TOPO=LIST" displays the topology of each node in the semicolon-
delimited list that is allocated to the job
* "CPUS[=LIST]" displays the available CPUs on the provided semicolon-
delimited list of nodes (defaults to all nodes)
The display command line directive can include qualifiers by adding a
colon (":") and any combination of one or more of the following
(delimited by colons):
* "PARSEABLE" directs that the output be provided in a format that is
easily parsed by machines. Note that "PARSABLE" is also accepted as
a typical spelling for the qualifier.
Provided qualifiers will apply to *all* of the display directives.
[runtime-options]
The "--runtime-options" command line directive must be accompanied by
a comma-delimited list of case-insensitive options that control the
runtime behavior of the job. The full directive need not be provided —
only enough characters are required to uniquely identify the
directive.
Runtime options are typically "true" or "false", though this is not a
requirement on developers. Since the value of each option may need to
be set (e.g., to override a default set by MCA parameter), the syntax
of the command line directive includes the use of an "=" character to
allow inclusion of a value for the option. For example, one can set
the "ABORT-NONZERO-STATUS" option to "true" by specifying it as
"ABORT-NONZERO-STATUS=1". Note that boolean options can be set to
"true" using a non-zero integer or a case-insensitive string of the
word "true". For the latter representation, the user need only
provide at least the "T" character. The same policy applies to setting
a boolean option to "false".
Note that a boolean option will default to "true" if provided without
a value. Thus, "--runtime-options abort-nonzero" is sufficient to set
the "ABORT-NONZERO-STATUS" option to "true".
Supported values include:
* "ERROR-NONZERO-STATUS[=(bool)]": if set to false, this directs the
runtime to treat a process that exits with non-zero status as a
normal termination. If set to true, the runtime will consider such
an occurrence as an error termination and take appropriate action —
i.e., the job will be terminated unless a runtime option directs
otherwise. This option defaults to a true value if the option is
given without a value.
* "DONOTLAUNCH": directs the runtime to map but not launch the
specified job. This is provided to help explore possible process
placement patterns before actually starting execution. No value need
be passed as this is not an option that can be set by default in
PRRTE.
* "SHOW-PROGRESS[=(bool)]": requests that the runtime provide progress
reports on its startup procedure — i.e., the launch of its daemons
in support of a job. This is typically used to debug DVM startup on
large systems. This option defaults to a true value if the option
is given without a value.
* "NOTIFYERRORS[=(bool)]": if set to true, requests that the runtime
provide a PMIx event whenever a job encounters an error — e.g., a
process fails. The event is to be delivered to each remaining
process in the job. This option defaults to a true value if the
option is given without a value. See "--help notifications" for
more detail as to the PMIx event codes available for capturing
failure events.
* "RECOVERABLE[=(bool)]": if set to true, this indicates that the
application wishes to consider the job as recoverable — i.e., the
application is assuming responsibility for recovering from any
process failure. This could include application-driven spawn of a
substitute process or internal compensation for the missing process.
This option defaults to a true value if the option is given without
a value.
* "AUTORESTART[=(bool)]": if set to true, this requests that the
runtime automatically restart failed processes up to "max restarts"
number of times. This option defaults to a true value if the option
is given without a value.
* "CONTINUOUS[=(bool)]": if set to true, this informs the runtime that
the processes in this job are to run until explicitly terminated.
Processes that fail are to be automatically restarted up to "max
restarts" number of times. Notification of process failure is to be
delivered to all processes in the application. This is the
equivalent of specifying "RECOVERABLE", "NOTIFYERRORS", and
"AUTORESTART" options except that the runtime, not the application,
assumes responsibility for process recovery. This option defaults to
a true value if the option is given without a value.
* "MAX-RESTARTS=<int>": indicates the maximum number of times a given
process is to be restarted. This can be set at the application or
job level (which will then apply to all applications in that job).
* "EXEC-AGENT=<path>" indicates the executable that shall be used to
start an application process. The resulting command for starting an
application process will be "<path> app <app-argv>". The path may
contain its own command line arguments.
* "DEFAULT-EXEC-AGENT": directs the runtime to use the system default
exec agent to start an application process. No value need be passed
as this is not an option that can be set by default in PRRTE.
* "OUTPUT-PROCTABLE[(=channel)]": directs the runtime to report the
convential debugger process table (includes PID and host location of
each process in the application). Output is directed to stdout if
the channel is "-", stderr if "+", or into the specified file
otherwise. If no channel is specified, output will be directed to
stdout.
* "STOP-ON-EXEC": directs the runtime to stop the application
process(es) immediately upon exec'ing them. The directive will apply
to all processes in the job.
* "STOP-IN-INIT": indicates that the runtime should direct the
application process(es) to stop in "PMIx_Init()". The directive will
apply to all processes in the job.
* "STOP-IN-APP": indicates that the runtime should direct application
processes to stop at some application-defined place and notify they
are ready-to-debug. The directive will apply to all processes in the
job.
* "TIMEOUT=<string>": directs the runtime to terminate the job after
it has executed for the specified time. Time is specified in colon-
delimited format — e.g., "01:20:13:05" to indicate 1 day, 20 hours,
13 minutes and 5 seconds. Time specified without colons will be
assumed to have been given in seconds.
* "SPAWN-TIMEOUT=<string>": directs the runtime to terminate the job
if job launch is not completed within the specified time. Time is
specified in colon-delimited format — e.g., "01:20:13:05" to
indicate 1 day, 20 hours, 13 minutes and 5 seconds. Time specified
without colons will be assumed to have been given in seconds.
* "REPORT-STATE-ON-TIMEOUT[(=bool)]": directs the runtime to provide a
detailed report on job and application process state upon job
timeout. This option defaults to a true value if the option is given
without a value.
* "GET-STACK-TRACES[(=bool)]": requests that the runtime provide stack
traces on all application processes still executing upon timeout.
This option defaults to a true value if the option is given without
a value.
* "REPORT-CHILD-JOBS-SEPARATELY[(=bool)]": directs the runtime to
report the exit status of any child jobs spawned by the primary job
separately. If false, then the final exit status reported will be
zero if the primary job and all spawned jobs exit normally, or the
first non-zero status returned by either primary or child jobs. This
option defaults to a true value if the option is given without a
value.
* "AGGREGATE-HELP-MESSAGES[(=bool)]": directs the runtime to aggregate
help messages, reporting each unique help message once accompanied
by the number of processes that reported it. This option defaults to
a true value if the option is given without a value.
* "FWD-ENVIRONMENT[(=bool)]": directs the runtime to forward the
entire local environment in support of the application. This option
defaults to a true value if the option is given without a value.
The "--runtime-options" command line option has no qualifiers.
Note:
Directives are case-insensitive. "FWD-ENVIRONMENT" is the same as
"fwd-environment".

276
macx64/mpi/openmpi/share/prte/help-prted.txt Normal file
View File

@@ -0,0 +1,276 @@
[bogus section]
This section is not used by PRTE code. But we have to put a RST
section title in this file somewhere, or Sphinx gets unhappy. So we
put it in a section that is ignored by PRTE code.
Hello, world
============
[usage]
%s (%s) %s
Usage: %s [OPTION]...
Start a PRTE Daemon
The following list of command line options are available. Note that
more detailed help for any option can be obtained by adding that
option to the help request as "--help <option>".
+----------------------+-----------------------------------------------+
| | General Options |
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "-h" | "--help" | This help message |
+----------------------+-----------------------------------------------+
| "-h" | "--help | Help for the specified option |
| <arg0>" | |
+----------------------+-----------------------------------------------+
| "-v" | "--verbose" | Enable typical debug options |
+----------------------+-----------------------------------------------+
| "-V" | "--version" | Print version and exit |
+----------------------+-----------------------------------------------+
+----------------------+-----------------------------------------------+
| | Debug Options |
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "--debug" | Top-level PRTE debug switch (default: false). |
| | This option will be deprecated, use "--debug- |
| | devel" instead. |
+----------------------+-----------------------------------------------+
| "--debug-daemons- | Enable debugging of any PRTE daemons used by |
| file" | this application, storing output in files |
+----------------------+-----------------------------------------------+
| "--leave-session- | Do not daemonize the "prted" |
| attached" | |
+----------------------+-----------------------------------------------+
| "--test-suicide | Suicide instead of clean abort after |
| <arg0>" | specified delay |
+----------------------+-----------------------------------------------+
+----------------------+-----------------------------------------------+
| | DVM Options |
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "--pmixmca <key> | Pass context-specific PMIx MCA parameters |
| <value>" | ("key" is the parameter name; "value" is the |
| | parameter value) |
+----------------------+-----------------------------------------------+
| "--prtemca <key> | Pass context-specific PRTE MCA parameters; |
| <value>" | they are considered global if "--gmca" is not |
| | used and only one context is specified ("key" |
| | is the parameter name; "value" is the |
| | parameter value) |
+----------------------+-----------------------------------------------+
| "--dvm-master-uri | Specify the URI of the DVM master, or the |
| <uri>" | name of the file (specified as |
| | "file:filename") that contains that info |
+----------------------+-----------------------------------------------+
| "--parent-uri <uri>" | Specify the URI of the prted acting as the |
| | parent of this prted in a tree-based spawn |
| | operation |
+----------------------+-----------------------------------------------+
| "--tree-spawn" | A tree-based spawn operation is in progress |
+----------------------+-----------------------------------------------+
| "--bootstrap" | Self-construct the DVM based on a |
| | configuration file |
+----------------------+-----------------------------------------------+
+----------------------+-----------------------------------------------+
| | Specific Options |
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "--set-sid" | Direct the DVM daemons to separate from the |
| | current session |
+----------------------+-----------------------------------------------+
| "--system-server" | Start the DVM as the system server |
+----------------------+-----------------------------------------------+
| "--pubsub-server | Contact information for external PRRTE |
| <uri>" | publish/lookup data server |
+----------------------+-----------------------------------------------+
Report bugs to %s
[version]
%s (%s) %s
Report bugs to %s
[dvm-master-uri]
Specify the URI of the DVM master, or the name of the file (specified
as "file:filename") that contains that info
[test-suicide]
Test DVM cleanup upon daemon failure by having one daemon suicide
after delay
[system-server]
Start the daemon as the system server on its node
[set-sid]
Direct the daemon to separate from the current session
[prtemca]
Pass a PRRTE MCA parameter.
Syntax: "--prtemca <key> <value>", where "key" is the parameter name
and "value" is the parameter value.
[pmixmca]
Pass a PMIx MCA parameter
Syntax: "--pmixmca <key> <value>", where "key" is the parameter name
and "value" is the parameter value.
[leave-session-attached]
Do not discard stdout/stderr of remote PRRTE daemons. The primary use
for this option is to ensure that the daemon output streams (i.e.,
stdout and stderr) remain open after launch, thus allowing the user to
see any daemon-generated error messages. Otherwise, the daemon will
"daemonize" itself upon launch, thereby closing its output streams.
[parent-uri]
Specify the URI of the prted acting as the parent of this prted in a
tree-based spawn operation
[tree-spawn]
A tree-based spawn operation is in progress
[pubsub-server]
Contact information for external PRRTE publish/lookup data server
%s
[prted:environ]
PRTE Daemon was unable to set
%s = %s
in the environment. Returned value %d instead of PRTE_SUCCESS.
[prted:init-failure]
PRTE was unable to initialize properly. The error occured while
attempting to %s. Returned value %d instead of PRTE_SUCCESS.
[prted:cannot-bind]
A request was made to bind the PRTE daemons to a core that does not
exist on this node:
node: %s
cores: %s
The MCA param directing this behavior is prte_daemon_cores. Please
correct the request and try again.
[cwd]
A dynamic operation (%s) was requested that requires us to obtain the
current working directory. Unfortunately, an error was returned when
we attempted to obtain it:
error: %d
We are unable to complete the requested operation.
[bad-key]
A dynamic operation (%s) was requested that included an unrecognized
info key:
group: %s
key: %s
The operation will continue, but may not behave completely as
expected.
[timedout]
A request has timed out and will therefore fail:
Operation: %s
Your job may terminate as a result of this problem. You may want to
adjust the MCA parameter pmix_server_max_wait and try again. If this
occurred during a connect/accept operation, you can adjust that time
using the pmix_base_exchange_timeout parameter.
[noroom]
A request for an asynchronous runtime operation cannot be fulfilled
because of a lack of room in the tracking array:
Operation: %s
Number of rooms: %d
This is usually caused by a large job that encounters significant
delays across the cluster when starting the application processes.
Your job may terminate as a result of this problem. You may want to
adjust the MCA parameter "pmix_server_max_reqs" and try again.
[noserver]
A publish/lookup server was provided, but we were unable to connect to
it — please check the connection info and ensure the server is alive:
Connection: %s
[mpir-debugger-detected]
PRRTE has detected that you have attached a debugger to this MPI job,
and that debugger is using the legacy "MPIR" method of attachment.
Please note that PRRTE has deprecated the "MPIR" debugger attachment
method in favor of the new "PMIx" debugger attchment mechanisms.
Warning:
This means that future versions of PRRTE may not support the "MPIR"
debugger attachment method at all. Specifically: the debugger you
just attached may not work with future versions of PRTE.
You may wish to contact your debugger vendor to inquire about support
for PMIx-based debugger attachment mechanisms. Meantime, you can
disable this warning by setting the "OMPI_MPIR_DO_NOT_WARN"
environment variable to 1.
[both-file-and-dir-set]
Both the "output-directory" and "output-filename" options have been
set:
Directory: %s
Filename: %s
Only one of these can be set — please fix the options and try again.
[min-pmix-violation]
PRRTE has detected that the PMIx library being used to run this
executable does not meet the minimum supported version:
Min PMIx version: %0x Detected version: %0x
Please check your LD_LIBRARY_PATH and ensure we are pointed to a
version that meets the minimum requirement.

2669
macx64/mpi/openmpi/share/prte/help-prterun.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1652
macx64/mpi/openmpi/share/prte/help-prun.txt Normal file
View File

File diff suppressed because it is too large Load Diff

318
macx64/mpi/openmpi/share/prte/help-psched.txt Normal file
View File

@@ -0,0 +1,318 @@
[bogus section]
This section is not used by PRTE code. But we have to put a RST
section title in this file somewhere, or Sphinx gets unhappy. So we
put it in a section that is ignored by PRTE code.
Hello, world
============
[usage]
%s (%s) %s
Usage: %s [OPTION]...
The following list of command line options are available. Note that
more detailed help for any option can be obtained by adding that
option to the help request as "--help <option>".
+----------------------+-----------------------------------------------+
| | General Options |
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "-h" | "--help" | This help message |
+----------------------+-----------------------------------------------+
| "-h" | "--help | Help for the specified option |
| <arg0>" | |
+----------------------+-----------------------------------------------+
| "-v" | "--verbose" | Enable typical debug options |
+----------------------+-----------------------------------------------+
| "-V" | "--version" | Print version and exit |
+----------------------+-----------------------------------------------+
| "--pmixmca <key> | Pass context-specific PMIx MCA parameters; |
| <value>" | they are considered global if only one |
| | context is specified ("key" is the parameter |
| | name; "value" is the parameter value) |
+----------------------+-----------------------------------------------+
| "--prtemca <key> | Pass context-specific PRTE MCA parameters; |
| <value>" | they are considered global if "--gmca" is not |
| | used and only one context is specified ("key" |
| | is the parameter name; "value" is the |
| | parameter value) |
+----------------------+-----------------------------------------------+
| "--tune <files>" | File(s) containing MCA params for tuning |
| | scheduler operations |
+----------------------+-----------------------------------------------+
+----------------------+-----------------------------------------------+
| | Resource Options |
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "--default-hostfile | Provide a default hostfile |
| <filename>" | |
+----------------------+-----------------------------------------------+
| "-H" | "--host | List of hosts to invoke processes on |
| <hosts>" | |
+----------------------+-----------------------------------------------+
| "--hostfile <file>" | Provide a hostfile |
+----------------------+-----------------------------------------------+
| "--machinefile | Provide a hostfile (synonym for "--hostfile") |
| <file>" | |
+----------------------+-----------------------------------------------+
+----------------------+-----------------------------------------------+
| | Specific Options |
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "--allow-run-as- | Allow execution as root (**STRONGLY |
| root" | DISCOURAGED**) |
+----------------------+-----------------------------------------------+
| "--daemonize" | Daemonize the scheduler into the background |
+----------------------+-----------------------------------------------+
| "--no-ready-msg" | Do not print a "DVM ready" message |
+----------------------+-----------------------------------------------+
| "--set-sid" | Direct the scheduler to separate from the |
| | current session |
+----------------------+-----------------------------------------------+
| "--tmpdir <dir>" | Set the filesystem root for the session |
| | directory tree |
+----------------------+-----------------------------------------------+
| "--report-pid <arg>" | Print out PID on stdout ("-"), stderr ("+"), |
| | or a file (anything else) |
+----------------------+-----------------------------------------------+
| "--report-uri <arg>" | Print out URI on stdout ("-"), stderr ("+"), |
| | or a file (anything else) |
+----------------------+-----------------------------------------------+
| "--keepalive <arg0>" | Pipe to monitor - scheduler will terminate |
| | upon closure |
+----------------------+-----------------------------------------------+
Report bugs to %s
[version]
%s (%s) %s
Report bugs to %s
[pmixmca]
Pass a PMIx MCA parameter
Syntax: "--pmixmca <key> <value>", where "key" is the parameter name
and "value" is the parameter value.
[prtemca]
Pass a PRRTE MCA parameter.
Syntax: "--prtemca <key> <value>", where "key" is the parameter name
and "value" is the parameter value.
[tune]
Comma-delimited list of one or more files containing PRRTE and PMIx
MCA params for tuning DVM and/or application operations. Parameters in
the file will be treated as *generic* parameters and subject to the
translation rules/uncertainties. See "--help mca" for more
information.
Syntax in the file is:
param = value
with one parameter and its associated value per line. Empty lines and
lines beginning with the "#" character are ignored, as is any
whitespace around the "=" character.
[default-hostfile]
Specify a default hostfile.
Also see "--help hostfile".
[host]
Host syntax consists of a comma-delimited list of node names, each
entry optionally containing a ":N" extension indicating the number of
slots to assign to that entry:
--host node01:5,node02
In the absence of the slot extension, one slot will be assigned to the
node. Duplicate entries are aggregated and the number of slots
assigned to that node are summed together.
Note:
A "slot" is the PRRTE term for an allocatable unit where we can
launch a process. Thus, the number of slots equates to the maximum
number of processes PRRTE may start on that node without
oversubscribing it.
[hostfile]
PRRTE supports several levels of user-specified host lists based on an
established precedence order. Users can specify a default hostfile
that contains a list of nodes to be used by the DVM. Only one default
hostfile can be provided for a given DVM. In addition, users can
specify a hostfile that contains a list of nodes to be used for a DVM,
or can provide a comma-delimited list of nodes to be used for that DVM
via the "--host" command line option.
The precedence order applied to these various options depends to some
extent on the local environment. The following table illustrates how
host and hostfile directives work together to define the set of hosts
upon which a DVM will execute in the absence of a resource manager
(RM):
+------------+---------+------------+----------------------------------+
| Default | host | hostfile | Result |
| hostfile | | | |
|============|=========|============|==================================|
| unset | unset | unset | The DVN will consist solely of |
| | | | the local host where the DVM was |
| | | | started. |
+------------+---------+------------+----------------------------------+
| unset | set | unset | Host option defines resource |
| | | | list for the DVM. |
+------------+---------+------------+----------------------------------+
| unset | unset | set | Hostfile option defines resource |
| | | | list for the DVM. |
+------------+---------+------------+----------------------------------+
| unset | set | set | Hostfile option defines resource |
| | | | list for the DVM, then host |
| | | | filters the list to define the |
| | | | final set of nodes to be used by |
| | | | the DVM |
+------------+---------+------------+----------------------------------+
| set | unset | unset | Default hostfile defines |
| | | | resource list for the DVM |
+------------+---------+------------+----------------------------------+
| set | set | unset | Default hostfile defines |
| | | | resource list for the DVM, then |
| | | | host filters the list to define |
| | | | the final set of nodes to be |
| | | | used by the DVM |
+------------+---------+------------+----------------------------------+
| set | set | set | Default hostfile defines |
| | | | resource list for the DVM, then |
| | | | hostfile filters the list, and |
| | | | then host filters the list to |
| | | | define the final set of nodes to |
| | | | be used by the DVM |
+------------+---------+------------+----------------------------------+
This changes somewhat in the presence of an RM as that entity
specifies the initial allocation of nodes. In this case, the default
hostfile, hostfile and host directives are all used to filter the RM's
specification so that a user can utilize different portions of the
allocation for different DVMs. This is done according to the same
precedence order as in the prior table, with the RM providing the
initial pool of nodes.
Hostfiles (sometimes called "machine files") are a combination of two
things:
1. A listing of hosts on which to launch processes.
2. Optionally, limit the number of processes which can be launched on
each host.
Hostfile syntax consists of one node name on each line, optionally
including a designated number of "slots":
# This is a comment line, and will be ignored
node01 slots=10
node13 slots=5
node15
node16
node17 slots=3
...
Blank lines and lines beginning with a "#" are ignored.
A "slot" is the PRRTE term for an allocatable unit where we can launch
a process. See the section on definition of the term "slot" for a
longer description of slots.
In the absence of the "slot" parameter, PRRTE will assign either the
number of slots to be the number of CPUs detected on the node or the
resource manager-assigned value if operating in the presence of an RM.
Important:
If using a resource manager, the user-specified number of slots is
capped by the RM-assigned value.
See the "Host specification" HTML documentation for details about the
format and content of hostfiles.
[machinefile]
Provide a hostfile. This option is a synonym for "--hostfile"; see
that option for more information.
[allow-run-as-root]
Allow execution as root **(STRONGLY DISCOURAGED)**.
Running as root exposes the user to potentially catastrophic file
system corruption and damage — e.g., if the user accidentally points
the root of the session directory to a system required point, this
directory and all underlying elements will be deleted upon job
completion, thereby rendering the system inoperable.
It is recognized that some environments (e.g., containers) may require
operation as root, and that the user accepts the risks in those
scenarios. Accordingly, one can override PRRTE's run-as-root
protection by providing one of the following:
* The "--allow-run-as-root" command line directive
* Adding **BOTH** of the following environmental parameters:
* "PRTE_ALLOW_RUN_AS_ROOT=1"
* "PRTE_ALLOW_RUN_AS_ROOT_CONFIRM=1"
Again, we recommend this only be done if absolutely necessary.
[daemonize]
Daemonize the scheduler into the background
[no-ready-msg]
Do not print a scheduler ready message
[set-sid]
Direct the scheduler to separate from the current session
[tmpdir]
Define the root location for the PRRTE session directory tree
See the "Session directory" HTML documentation for additional details
about the PRRTE session directory.
[report-pid]
Printout scheduler's PID on stdout [-], stderr [+], or a file
[anything else]
[report-uri]
Printout scheduler's URI on stdout [-], stderr [+], or a file
[anything else]
[keepalive]
Pipe for scheduler to monitor - scheduler will terminate upon closure

119
macx64/mpi/openmpi/share/prte/help-pterm.txt Normal file
View File

@@ -0,0 +1,119 @@
[bogus section]
This section is not used by PRTE code. But we have to put a RST
section title in this file somewhere, or Sphinx gets unhappy. So we
put it in a section that is ignored by PRTE code.
Hello, world
============
[usage]
%s (%s) %s
Usage: %s [OPTION]...
Terminate an instance of the PMIx Reference RTE (PRRTE) DVM
* General Options
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "-h" | "--help" | This help message |
+----------------------+-----------------------------------------------+
| "-h" | "--help | Help for the specified option |
| <arg0>" | |
+----------------------+-----------------------------------------------+
| "-v" | "--verbose" | Enable typical debug options |
+----------------------+-----------------------------------------------+
| "-V" | "--version" | Print version and exit |
+----------------------+-----------------------------------------------+
* Specific Options
+----------------------+-----------------------------------------------+
| Option | Description |
|======================|===============================================|
| "--pmixmca <key> | Pass context-specific PMIx MCA parameters |
| <value>" | ("key" is the parameter name; "value" is the |
| | parameter value) |
+----------------------+-----------------------------------------------+
| "--dvm-uri <uri>" | Specify the URI of the DVM master, or the |
| | name of the file (specified as |
| | "file:filename") that contains that info |
+----------------------+-----------------------------------------------+
| "--num-connect- | Max number of times to try to connect (int) |
| retries <num>" | |
+----------------------+-----------------------------------------------+
| "--pid <pid>`" | PID of the daemon to which we should connect |
| | (integer PID or "file:<filename>" for file |
| | containing the PID |
+----------------------+-----------------------------------------------+
| "--namespace <name>" | Namespace of the daemon we are to connect to |
+----------------------+-----------------------------------------------+
| "--system-server- | First look for a system server and connect to |
| first" | it if found |
+----------------------+-----------------------------------------------+
| "--system-server- | Connect only to a system-level server |
| only" | |
+----------------------+-----------------------------------------------+
| "--wait-to-connect | Delay specified number of seconds before |
| <seconds>" | trying to connect |
+----------------------+-----------------------------------------------+
Report bugs to %s
[version]
%s (%s) %s
Report bugs to %s
[dvm-uri]
Specify the URI of the DVM master, or the name of the file (specified
as "file:<filename>") that contains that info
[num-connect-retries]
Max number of times to try to connect to the specified server (int)
[pid]
PID of the daemon to which we should connect (integer PID or
"file:<filename>" for file containing the PID
[namespace]
Namespace of the daemon we are to connect to (char*)
[system-server-first]
First look for a system server and connect to it if found
[system-server-only]
Connect only to a system-level server - abort if one is not found
[wait-to-connect]
Delay specified number of seconds before trying to connect
[pmixmca]
Pass a PMIx MCA parameter
Syntax: "--pmixmca <key> <value>", where "key" is the parameter name
and "value" is the parameter value.
[no-args]
The %s command does not accept arguments other than those specifically
defined by the command. The following were not recognized:
Args: %s
Please see "%s --help" for a description of all accepted command
options.

64
macx64/mpi/openmpi/share/prte/help-ras-base.txt Normal file
View File

@@ -0,0 +1,64 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2024 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2019 Intel, Inc. All rights reserved.
# Copyright (c) 2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2023-2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English general help file for the RAS base.
#
[ras-base:no-allocation]
We were unable to find an allocation for this job as required by
setting the "allocation required" flag. Please ensure you have
the necessary allocation before executing again.
If you wish to execute without a provided allocation (e.g., by
providing a user-specified hostfile), please ensure that the "allocation
required" flag is not set. This flag can be set in several forms, so
please check that none of the following exist:
MCA param file: prte_allocation_required = 1
Environment: PRTE_MCA_prte_allocation_required=1
Cmd line: -prtemca prte_allocation_required 1
#
[ras-sim:mismatch]
The number of topology files and the list of number of nodes
must match - i.e., a number of nodes must be given for each
topology.
#
[ras-base:addhost-not-found]
A request was made to add hosts from a hostfile, but the
file could not be opened for reading:
File: %s
Please check the filename and try again.
#
[ras-base:nonuniform-slots]
A request was made to add hosts from a hostfile while operating
in a managed allocation. In this case, either the slots must be
specified in the given hostfile, or the number of slots assigned
by the resource manager on the existing nodes must be uniform.
The current allocation does not conform to that requirement:
Base number of slots: %d
Node: %s
Number of slots: %d
Please assign a number of slots for each node to be added to the
allocation.

43
macx64/mpi/openmpi/share/prte/help-ras-pbs.txt Normal file
View File

@@ -0,0 +1,43 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2006 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
# Copyright (c) 2014-2019 Intel, Inc. All rights reserved.
# Copyright (c) 2022-2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
[no-nodefile]
The PBS_NODEFILE environmental variable was
not found - no nodes can be allocated
[no-nodes-found]
No nodes were found in the PBS_NODEFILE:
%s
This is probably a setup issue, could be due to a corrupted file, or
- most probably - is caused by having failed to obtain an allocation.
#
[smp-error]
The SMP mode flag was given, but the required PBS_PPN environmental
variable was not found. Please check that your system actually supports
the indicated mode and try again.
#
[smp-multi]
The SMP mode flag was given, but a node was listed in the PBS_Nodefile
more than once. This usually indicates that the system is not
actually operating in SMP mode. Please check your system configuration
and try again.

35
macx64/mpi/openmpi/share/prte/help-ras-simulator.txt Normal file
View File

@@ -0,0 +1,35 @@
# -*- text -*-
#
# Copyright (c) 2012-2020 Cisco Systems, Inc. All rights reserved
#
# Copyright (c) 2019-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
[no hwloc support for topofiles]
The ras_simulator_topo_files MCA parameter was specified, but this
PRTE installation has no "hwloc" support, meaning that topo files
cannot be used.
Your job will now abort.
#
[hwloc API fail]
An internal hwloc API call failed in the RAS simulator module. This
is unusual, and should likely be reported to the PRTE developers.
Your job will now abort.
File: %s
Line: %d
API: %s
#
[hwloc failed to load xml]
The RAS simulator module failed to load an XML topology file. This
usually means that the file is either invalid or does not exist.
Topo file: %s
#

81
macx64/mpi/openmpi/share/prte/help-ras-slurm.txt Normal file
View File

@@ -0,0 +1,81 @@
# -*- text -*-
#
# Copyright (c) 2004-2007 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2019-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English help file for PRTE MCA error messages.
#
[slurm-env-var-not-found]
While trying to determine what resources are available, the SLURM
resource allocator expects to find the following environment variables:
SLURM_NODELIST
SLURM_TASKS_PER_NODE
However, it was unable to find the following environment variable:
%s
#This is a fatal error.
[slurm-env-var-bad-value]
While trying to determine what resources are available, the SLURM
resource allocator uses the following environment variables:
SLURM_NODELIST value: %s
SLURM_TASKS_PER_NODE value: %s
However, an error was encountered when trying to parse the following variable:
%s
#This is a fatal error.
[slurm-dyn-alloc-timeout]
We attempted to obtain a dynamic allocation from Slurm, but
contact with the Slurm control daemon timed out. Please check
that the Slurm dynamic allocation plug-in is properly operating.
#
[slurm-dyn-alloc-failed]
Allocation request: %s
#
[dyn-alloc-no-config]
Dynamic allocation was enabled, but no Slurm configuration
file was given. Please provide the required configuration file.
#
[host-not-resolved]
The Slurm control host could not be resolved.
Host: %s
Please check the Slurm configuration and try again.
#
[connection-failed]
Connection to the Slurm controller failed:
Host: %s
Port: %d
Please check the Slurm configuration and try again.
#
[config-file-not-found]
The Slurm configuration file was not found.
File: %s
Please check the filename and path and try again.

110
macx64/mpi/openmpi/share/prte/help-rmaps_rank_file.txt Normal file
View File

@@ -0,0 +1,110 @@
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2011-2020 Cisco Systems, Inc. All rights reserved
# Copyright (c) 2013 Los Alamos National Security, LLC.
# All rights reserved.
# Copyright (c) 2018-2020 Intel, Inc. All rights reserved.
# Copyright (c) 2021-2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This is the US/English general help file for rankle utilities.
#
[no-rankfile]
%s was unable to open the rankfile:
Filename: %s
Check to make sure the path and filename are correct.
usage: %s -mca rmaps_rankfile_path rankfile ./app
Examples of proper syntax include:
cat hostfile
host1
host2
host3
host4
cat rankfile
rank 1=host1 slot=1:0,1
rank 0=host2 slot=0:*
rank 2=host4 slot=1-2
rank 3=host3 slot=0:1;1:0-2
#
[bad-host]
The rankfile that was used claimed that a host was either not
allocated or oversubscribed its slots. Please review your rank-slot
assignments and your host allocation to ensure a proper match. Also,
some systems may require using full hostnames, such as
"host1.example.com" (instead of just plain "host1").
Host: %s
#
[bad-index]
Rankfile claimed host %s by index that is bigger than number of allocated hosts.
#
[bad-assign]
Error, rank %d is already assigned to %s, check %s
#
[bad-syntax]
Error, invalid syntax in the rankfile (%s)
syntax must be the following
rank i=host_i slot=string
Examples of proper syntax include:
rank 1=host1 slot=1:0,1
rank 0=host2 slot=0:*
rank 2=host4 slot=1-2
rank 3=host3 slot=0:1;1:0-2
#
[missing-rank]
A rank is missing its location specification:
Rank: %d
Rank file: %s
All processes must have their location specified in the rank file. Either
add an entry to the file, or provide a default slot_list to use for
any unspecified ranks.
#
[missing-cpu]
While parsing the rankfile, %s encountered a line that specified
a non-existent CPU:
Slots: %s
Available CPUs: %s
Please correct the line and try again.
[lsf-affinity-file-not-found]
The affinity file provided in LSB_AFFINITY_HOSTFILE could not be found:
File: %s
We cannot continue.
[lsf-affinity-file-failed-convert]
The affinity file provided in LSB_AFFINITY_HOSTFILE could not be converted
to a rankfile:
File: %s
We cannot continue.
[rmaps:proc-slots-overloaded]
A process attempted to bind to resources already allocated on a node
to a different process.
Process : %s
Node : %s
Process Requested CPU Set : %s
Node Available CPU Set : %s
Overlapping CPU Set : %s
If this is intentional then you must pass the "overload-allowed"
qualifier to the --bind-to option.
--bind-to :overload-allowed
#
[resource-not-found]
The specified LSF affinity file contained a node (%s) that is not in your
allocation. We therefore cannot map a process rank to it. Please
check your allocation and affinity file to ensure the latter only
contains allocated nodes.

195
macx64/mpi/openmpi/share/prte/help-schizo-base.txt Normal file
View File

@@ -0,0 +1,195 @@
# -*- text -*-
#
# Copyright (c) 2020 Intel, Inc. All rights reserved.
# Copyright (c) 2020 IBM Corporation. All rights reserved.
# Copyright (c) 2021-2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
#
[missing-param-file]
Unable to locate the variable file:
%s
[missing-param-file-def]
Unable to locate the variable file:
%s
either relative to the current working directory
or in the default location:
%s
#
[bad-param-line]
Parsing error in variable file:
FILE: %s
LINE: %s
Please correct and try again.
#
[incorrect-env-list-param]
WARNING: The format of "mca_base_env_list" parameter is a delimited list of VAR=VAL or
VAR instances. By default, the delimiter is a semicolon: VAR1=VAL1;VAR2;VAR3=VAL3;...
You can set other via "mca_base_env_list_delimiter" parameter. If the delimiter is a
semicolon, the value of "mca_base_env_list" variable should be quoted to not interfere
with SHELL command line parsing. In the case where a value is not assigned to variable
VAR, the value will be taken from the current environment.
The following environment variable was not found in the environment:
Variable: %s
MCA variable value: %s
#
[env-not-found]
An environmental variable was requested to be forwarded, but was not found:
Variable: %s
This may result in unexpected behavior.
#
[single-dash-error]
WARNING: All multi-character command line options must be prefixed with double
dashes, not a single dash:
Deprecated option(s): %s
Corrected option(s): %s
We have updated this for you and will proceed. However, this will be treated
as an error in a future release. Please update your command line.
#
[deprecated-converted]
WARNING: A deprecated command line option was used.
Deprecated option: %s
Corrected option: %s
We have updated this for you and will proceed. However, this will be treated
as an error in a future release. Please update your command line.
#
[deprecated-inform]
WARNING: A deprecated command line option was used.
Deprecated option: %s
Reason: %s
#
[deprecated-fail]
WARNING: A deprecated command line option was used. It is unclear how to update
this option. Please contact the community for how best to resolve this issue.
Deprecated option: %s
Reason: %s
We are not able to proceed. Please update your command line.
#
[bad-ppr]
The proc-per-resource request is not correct. The request must include both
the number of procs for each resource and the type of resource to be used.
Specified option: %s
We are not able to proceed. Please correct your command line.
#
[bad-stream-buffering-value]
An incorrect value for the "--stream-buffering" option was given:
Specified value: %d
Valid values are limited to 0, 1, or 2. Your application will continue, but
please correct your command line in the future.
#
[missing-envar-param]
Warning: Could not find the environment variable: %s
#
[duplicate-mca-value]
Error: an MCA parameter was listed more than once on the command line, or
multiple times in one or more files, but with conflicting values:
Param: %s
Value: %s
Value: %s
We cannot determine which value was intended, and therefore are unable to proceed.
Please correct your command line.
#
[multi-instances]
ERROR: The "%s" command line option was listed more than once on the command line.
Only one instance of this option is permitted.
Please correct your command line.
#
[no-proxy]
ERROR: %s was unable to identify the proxy for your command.
Option specified: %s
Please specify a personality or ask the library developer's to address
the problem.
#
[unrecog-generic-param]
The command line contains a generic MCA param that is not
recognized by either the default or active personalities:
Param name: %s
Param value: %s
Please modify the command line to either specify the personality
this parameter belongs to (e.g., "--prtemca" for a PRRTE param,
"--pmixmca" for a PMIx param, "--omca" for an OMPI param) or
ensure that the generic param is correct and known to the active
personality.
#
[too-many-instances]
The command line includes too many instances of the following option:
Command line: %s
Option: %s
Number of instances: %d
Allowed number: %d
Please correct the command line and try again.
#
[too-many-directives]
When resolving deprecated command line options, the result generated
too many directives for the target option:
Command line option: %s
Initial value: %s
Deprecated option: %s %s
Please correct the command line and try again.
#
[too-many-values]
When resolving deprecated command line options, the following
command line option has more than one value:
Option: %s
The resolution function cannot handle this scenario - please
notify the developers
#
[binding-pe-conflict]
The PE=<list> mapping directive cannot be combined with a
binding directive other than "core" or "hwt" as it already
mandates that we bind to the specified cpu(s). The conflicting
directives that were given are:
map-by: %s
bind-to: %s
Please resolve the conflict and try again.
#
[missing-values]
The provided command line option requires passing two
values:
Option: %s <MCA param name> <param value>
Please provide the missing values and try again.
#
[non-supporting-pmix]
The given command line directive is technically valid, but the
underlying PMIx version being employed lacks the necessary attribute
definitions to support it.
Directive: %s
Option: %s
We are therefore unable to satisfy this request.

471
macx64/mpi/openmpi/share/prte/help-schizo-ompi.txt Normal file
View File

@@ -0,0 +1,471 @@
# -*- text -*-
#
# Copyright (c) 2021-2024 Nanook Consulting All rights reserved.
# Copyright (c) 2022 Cisco Systems, Inc. All rights reserved.
# Copyright (c) 2022 IBM Corporation. All rights reserved.
# Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
#
[version]
%s (%s) %s
%s
#
[usage]
%s (%s) %s
Usage: %s [OPTION]...
See the mpirun(1) man page or HTML help for a detailed list of command
line options that are available.
%s
#
[dvm]
Utilize an existing persistent DVM instead of instantiating an
independent runtime infrastructure. This is provided as a convenience
option for those wanting to embed the "mpirun" command in a script that
can be optionally used to run either independently or under a persistent DVM.
#include#help-schizo-cli#dvm
#
[prtemca]
Syntax: --prtemca <arg0> <arg1>
where arg0 is the parameter name and arg1 is the parameter value
Pass a PRRTE MCA parameter
#
[pmixmca]
Syntax: --pmixmca <arg0> <arg1>
where arg0 is the parameter name and arg1 is the parameter value
Pass a PMIx MCA parameter
#
[tune]
File(s) containing PRRTE and PMIx MCA params for tuning DVM and/or application operations.
Parameters in the file will be treated as _generic_ parameters and subject to the
translation rules/uncertainties. See "--help mca" for more information.
Syntax in the file is:
param=value
with one parameter and its associated value per line. Empty lines and lines beginning
with the '#' character are ignored.
There must not be any whitespace characters before or after the '=' in the param/value
specification.
#
[daemonize]
Daemonize the DVM daemons into the background
#
[set-sid]
Direct the DVM daemons to separate from the current session
#
[report-pid]
Display the PID a tool can use to connect to the mpirun process on stdout (-), stderr (+), or write it to
the specified file if anything else is specified.
#
[report-uri]
Display the URI that a tool can use to connect to the mpirun process on stdout (-), stderr (+), or write
it to a file if anything else is specified.
#
[test-suicide]
Suicide instead of clean abort after delay
#
[default-hostfile]
This option specifies a default set of nodes that are to be used.
If neither the --hostfile or --host options are specified, this will be the set of nodes used.
The --hostfile and --host options can be used to select a subset of nodes from the default set to be used.
If both the --hostfile and --host options are specified, then the --hostfile option specifies the
initial subset of nodes used from the default-hostfile set, and the --host option specifies what subset of
nodes from that second subset will be used as the final subset of nodes used.
If a resource manager is used, the resource manager determines the initial set of nodes. Then the
specifications for the --default-hostfile, --hostfile, and --host optons, in that order, are used to determine
the final subset of nodes that are used.
#
[singleton]
ID of the singleton process that started us
#
[keepalive]
Pipe to monitor - DVM will terminate upon closure of this pipe.
#
[launch-agent]
Name of daemon executable used to start processes on remote nodes (default: prted)
#
[max-vm-size]
Maximum number of daemons to start
#
[debug-daemons]
Debug daemon output enabled
#
[debug-daemons-file]
Enable debugging of any PRTE daemons used by this application, storing output in files
#
[leave-session-attached]
Do not discard stdout/stderr of remote PRTE daemons, mostly used when debugging the
application launcher.
#
[tmpdir]
Set the root for the session directory tree
#
[prefix]
This option specifies the path to be used as a prefix when resolving the path to Open MPI
executables.
#
[noprefix]
Ignores the path specified if Open MPI was configured with the --prefix option when built
from source code.
#
[forward-signals]
Comma-delimited list of additional signals (names or integers) to forward to application
processes ["none" => forward nothing]. Signals provided by default include SIGTSTP,
SIGUSR1, SIGUSR2, SIGABRT, SIGALRM, and SIGCONT
#
[allow-run-as-root]
Allow execution as root (STRONGLY DISCOURAGED)
#
[report-child-jobs-separately]
Return the exit status of the primary job only
#
[timeout]
Timeout the job if execution time exceeds the specified number of seconds
#
[report-state-on-timeout]
Report all job and process states upon timeout
#
[get-stack-traces]
Get stack traces of all application procs on timeout
#
[spawn-timeout]
Timeout the job if spawn takes more than the specified number of seconds
#
[np]
Specify number of application processes to be started
#
[n]
Specify number of application processes to be started
#
[N]
Specify number of application processes per node to be started
#
[app]
Provide an appfile; ignore all other command line options
#
[xterm]
Create a new xterm window and display output from the specified ranks there.
Ranks are specified as a comma-delimited list of ranges - e.g., "1,3-6,9",
or as "all"."
#
[stop-on-exec]
If supported, stop each process at start of execution
#
[stop-in-init]
Include the PMIX_DEBUG_STOP_IN_INIT attribute in the application's
job info directing that the ranks stop in PMIx_Init pending
release.
#
[stop-in-app]
Include the PMIX_DEBUG_STOP_IN_APP attribute in the application's
job info directing that the ranks stop at an application-determined
point pending release.
#
[x]
Export an environment variable, optionally specifying a value (e.g., "-x foo" exports the
environment variable foo and takes its value from the current environment; "-x foo=bar"
exports the environment variable name foo and sets its value to "bar" in the started
processes; "-x foo*" exports all current environmental variables starting with "foo")
#
[wdir]
Set the working directory of the started processes
#
[wd]
Synonym for --wdir
#
[set-cwd-to-session-dir]
Set the working directory of the started processes to their session directory
#
[path]
PATH to be used to look for executables to start processes
#
[show-progress]
Output a brief periodic report on launch progress
#
[pset]
User-specified name assigned to the processes in their given application
#
[hostfile]
Provide a hostfile
#
[machinefile]
Provide a hostfile (synonym for "hostfile")
#
[host]
Comma-separated list of hosts to invoke processes on
#
[personality]
Specify the personality to be used
#
[preload-files]
Syntax: --preload-files <arg0>
Preload the comma separated list of files to the remote machines current
working directory before starting the remote process.
#
[preload-binary]
Syntax: --preload-binary
Preload the binary on the remote machine before starting the
remote process.
#
[output]
Comma-delimited list of case-insensitive options that control how output is generated.
Allowed values:
tag Mark each output line with the [job,rank] of the
process that generated it
timestamp Prefix each output line with a datetime stamp. Note
that the timestamp will be the time when the line
is output by the DVM - not the time when the source
output it
xml Format all output in XML
merge[-stderr-to-stdout] Merge stderr into stdout
dir=DIRNAME Redirect output from application processes into files of
the form DIRNAME/job/rank/std[out,err,diag].
file=filename Redirect output from application processes into files of
the form filename.rank.
In both the "dir" and "file" cases, the provided name will be converted to an absolute path.
Supported qualifiers include NOCOPY (i.e., output shall go only into the files - do not copy
the output to the stdout/err streams).
Only enough characters are required to uniquely identify the option.
For example, TI is sufficient to identify TIMESTAMP option, while T cannot be used
since it does not uniquely identify an option.
#
[stream-buffering]
Adjust buffering for stdout/stderr [0 unbuffered] [1 line buffered] [2 fully buffered]
Can also be set via the MCA parameter: --mca ompi_stream_buffering VALUE
#
[stdin]
Specify procs to receive stdin [rank, "all", "none"] (default: 0, indicating rank 0)
#
[map-by]
Mapping Policy for job:
slot
hwthread
core (default: np <= 2)
l1cache
l2cache
l3cache
numa (default: np > 2)
package
node
seq
dist
ppr
rankfile
with supported colon-delimited qualifiers:
PE=y (for multiple cpus/proc)
SPAN
OVERSUBSCRIBE
NOOVERSUBSCRIBE
NOLOCAL
HWTCPUS
CORECPUS
DEVICE(for dist policy)
INHERIT
NOINHERIT
PE-LIST=a,b (comma-delimited ranges of cpus to use for this job)
FILE=<path> for seq and rankfile options
#
[rank-by]
Ranking Policy for job:
slot (default: np <= 2)
hwthread
core
l1cache
l2cache
l3cache
numa (default: np > 2)
package
node
with supported colon-delimited qualifiers:
SPAN
FILL
#
[bind-to]
Binding Policy for job:
none (default: oversubscribed)
hwthread
core (default: np <= 2)
l1cache
l2cache
l3cache
numa (default: np > 2)
package
with supported colon-delimited qualifiers:
overload-allowed
if-supported
#
[rankfile]
Name of file to specify explicit task mapping
#
[display]
Comma-delimited list of case-insensitive options for displaying information about
the allocation and job.
Allowed values:
allocation
bind
map
map-devel
topo
Only enough characters are required to uniquely identify the option.
For example, A is sufficient to identify ALLOCATION option, while MAP cannot be used
to specify the MAP-DEVEL option since there is also a MAP option.
#
[do-not-launch]
Perform all necessary operations to prepare to launch the application, but do not actually
launch it (usually used to test mapping patterns)
#
[enable-recovery]
Enable recovery from process failure [Default = disabled]
#
[max-restarts]
Max number of times to restart a failed process
#
[disable-recovery]
Disable recovery (resets all recovery options to off)
#
[continuous]
Job is to run until explicitly terminated
[openmpi-install-path-not-found]
An internal Open MPI environment variable needed for Java support was not set.
Open MPI 5.0.0 or newer is required for this support.
#
[output-proctable]
Print the complete proctable to stdout after launch. This option accepts an optional argument,
which must be + to print to stdout, - to print to stderr, or a valid filename to print to
a file. For instance --output-proctable=+
If the optional argument is omitted, then the table is printed to stdout.
The table includes rank, hostname, executable name and pid.
#
# DEPRECATED OPTIONS
#
[mca]
Syntax: --mca <arg0> <arg1>
where arg0 is the parameter name and arg1 is the parameter value
Pass generic MCA parameters - i.e., parameters whose project affiliation
must be determined by PRRTE based on matching the name of the parameter with defined values
from various projects that PRRTE knows about.
DEPRECATED: This translation can be incomplete (e.g., if known project adds or changes
parameters) - thus, it is strongly recommended that users use project-specific parameters
such as "prtemca" or "pmixmca".
#
[gmca]
Syntax: --gmca <arg0> <arg1>
where arg0 is the parameter name and arg1 is the parameter value. The "g" prefix
indicates that this parameter is to be applied to _all_ application contexts and
not just the one in which the directive appears.
Pass generic MCA parameters - i.e., parameters whose project affiliation
must be determined by PRRTE based on matching the name of the parameter with defined values
from various projects that PRRTE knows about. This translation can be incomplete (e.g., if
a known project adds or changes parameters) - thus, it is strongly recommended that users
use project-specific parameters such as "prtemca" or "pmixmca".
DEPRECATED: This translation can be incomplete (e.g., if known project adds or changes
parameters) - thus, it is strongly recommended that users use project-specific parameters
such as "gprtemca" or "gpmixmca".
#
[xml]
Provide all output in XML format
DEPRECATED: please see "--help output" for details
#
[tag-output]
Tag all output with [job,rank]
DEPRECATED: please see "--help output" for details
#
[timestamp-output]
Timestamp all application process output
DEPRECATED: please see "--help output" for details
#
[output-directory]
Redirect output from application processes into filename/job/rank/std[out,err,diag]. A
relative path value will be converted to an absolute path. The directory name may include a
colon followed by a comma-delimited list of optional case-insensitive directives. Supported
directives currently include NOJOBID (do not include a job-id directory level) and NOCOPY
(do not copy the output to the stdout/err streams)
DEPRECATED: please see "--help output" for details
#
[output-filename]
Redirect output from application processes into filename.rank. A relative path value will be
converted to an absolute path. The directory name may include a colon followed by a
comma-delimited list of optional case-insensitive directives. Supported directives currently
include NOCOPY (do not copy the output to the stdout/err streams)
DEPRECATED: please see "--help output" for details
#
[merge-stderr-to-stdout]
Merge stderr to stdout for each process
DEPRECATED: please see "--help output" for details
#
[display-devel-map]
Display a detailed process map (mostly intended for developers)
just before launch
DEPRECATED: please see "--help display" for details
#
[display-topo]
Display the topology as part of the process map (mostly intended
for developers) just before launch
DEPRECATED: please see "--help display" for details
#
[report-bindings]
Display process bindings to stderr
DEPRECATED: please see "--help display" for details
#
[display-map]
Display the process map just before launch
DEPRECATED: please see "--help display" for details
#
[display-allocation]
Display the allocation being used by this job
DEPRECATED: please see "--help display" for details
#
[with-ft-bad-option]
The OMPI personality was passed the --with-ft directive,
but that directive was given an unsupported option:
Option: %s
Supported options include:
* typical represenations for "true" ("yes", "true", or "1")
* typical representations for "false" ("no", "false", or "0")
* either "ulfm" or "mpi"
Please provide one of the above values and try again.

59
macx64/mpi/openmpi/share/prte/help-state-base.txt Normal file
View File

@@ -0,0 +1,59 @@
# -*- text -*-
#
# Copyright (c) 2018 Research Organization for Information Science
# and Technology (RIST). All rights reserved.
# Copyright (c) 2019 Intel, Inc. All rights reserved.
# Copyright (c) 2022-2024 Nanook Consulting All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
#
[normal-termination-but]
While %s job %s terminated normally, %d %s. Further examination may be required.
#
[bad-combination]
The runtime option NOTIFYERRORS was set to true. This requests that the
runtime provide a PMIx event whenever a job encounters an error - e.g., a
process fails. The event is to be delivered to each remaining process in
the job. However, this only can be done if the job is not to be terminated
in the event of an error. The following runtime options govern that
behavior:
RECOVERABLE[=(bool)] if set to true, this indicates that the application
wishes to consider the job as recoverable - i.e., the application is
assuming responsibility for recovering from any process failure. This
could include application-driven spawn of a substitute process or
internal compensation for the missing process. This option defaults to
a true value if the option is given without a value.
CONTINUOUS[=(bool)] if set to true, this informs the runtime that the
processes in this job are to run until explicitly terminated. Processes
that fail are to be automatically restarted up to "max restarts"
number of times. Notification of process failure is to be delivered
to all processes in the application. This is the equivalent of specifying
RECOVERABLE, NOTIFYERRORS, and AUTORESTART options except that the runtime,
not the application, assumes responsibility for process recovery. This
option defaults to a true value if the option is given without a value.
You must specify one of the above in combination with NOTIFYERRORS in order
to receive notifications of errors. Please correct the situation and try again.
#
[child-term]
At least one child job is being terminated due to termination of
its parent:
Parent: %s
Child: %s
This behavior is controlled by setting the PMIX_JOB_CHILD_SEP attribute
in the job info provided at time of spawn for the child job. When set to
"true", the runtime will "separate" the child from its parent and allow
it to continue execution after parent termination. Note that this is only
true for parents that normally terminate - abnormal termination will always
result in a complete teardown of all child jobs.
In the absence of the attribute, the runtime will default to the "true"
behavior.

34
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-allow-run-as-root.rst Normal file
View File

@@ -0,0 +1,34 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Allow execution as root **(STRONGLY DISCOURAGED)**.
Running as root exposes the user to potentially catastrophic file
system corruption and damage |mdash| e.g., if the user accidentally
points the root of the session directory to a system required point,
this directory and all underlying elements will be deleted upon job
completion, thereby rendering the system inoperable.
It is recognized that some environments (e.g., containers) may require
operation as root, and that the user accepts the risks in those
scenarios. Accordingly, one can override PRRTE's run-as-root
protection by providing one of the following:
* The ``--allow-run-as-root`` command line directive
* Adding **BOTH** of the following environmental parameters:
* ``PRTE_ALLOW_RUN_AS_ROOT=1``
* ``PRTE_ALLOW_RUN_AS_ROOT_CONFIRM=1``
Again, we recommend this only be done if absolutely necessary.

83
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-bind-to.rst Normal file
View File

@@ -0,0 +1,83 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
By default, processes are bound to individual CPUs (either COREs or
HWTHREADs, as defined by default or by user specification for the
job). On nodes that are OVERSUBSCRIBEd (i.e., where the number of
procs exceeds the number of assigned slots), the default is to not
bind the processes.
.. note:: Processes from prior jobs that are already executing on a
node are not "unbound" when a new job mapping results in the
node becoming oversubscribed.
Binding is performed to the first available specified object type
within the object where the process was mapped. In other words,
binding can only be done to the mapped object or to a resource
located beneath that object.
An object is considered completely consumed when the number of
processes bound to it equals the number of CPUs within it. Unbound
processes are not considered in this computation. Additional
processes cannot be mapped to consumed objects unless the
``OVERLOAD`` qualifier is provided via the ``--bind-to`` command
line option.
Note that directives and qualifiers are case-insensitive
and can be shortened to the minimum number of characters
to uniquely identify them. Thus, ``L1CACHE`` can be given
as ``l1cache`` or simply as ``L1``.
Supported binding directives include:
* ``NONE`` does not bind the processes
* ``HWTHREAD`` binds each process to a single hardware
thread/ This requires that hwthreads be treated
as independent CPUs (i.e., that either the ``HWTCPUS``
qualifier be provided to the ``map-by`` option or
that ``hwthreads`` be designated as CPUs by default).
* ``CORE`` binds each process to a single core. This
can be done whether ``hwthreads`` or ``cores`` are being
treated as independent CPUs provided that mapping
is performed at the core or higher level.
* ``L1CACHE`` binds each process to all the CPUs in
an ``L1`` cache.
* ``L2CACHE`` binds each process to all the CPUs in
an ``L2`` cache
* ``L3CACHE`` binds each process to all the CPUs in
an ``L3`` cache
* ``NUMA`` binds each process to all the CPUs in a ``NUMA``
region
* ``PACKAGE`` binds each process to all the CPUs in a ``PACKAGE``
Any directive can include qualifiers by adding a colon (:) and any
combination of one or more of the following to the ``--bind-to``
option:
* ``OVERLOAD`` indicates that objects can have more
processes bound to them than CPUs within them
* ``IF-SUPPORTED`` indicates that the job should continue to
be launched and executed even if binding cannot be
performed as requested.
.. note:: Directives and qualifiers are case-insensitive.
``OVERLOAD`` is the same as ``overload``.

30
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-dash-host.rst Normal file
View File

@@ -0,0 +1,30 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Host syntax consists of a comma-delimited list of node names, each
entry optionally containing a ``:N`` extension indicating the number
of slots to assign to that entry:
.. code::
--host node01:5,node02
In the absence of the slot extension, one slot will be assigned to the
node. Duplicate entries are aggregated and the number of slots
assigned to that node are summed together.
.. note:: A "slot" is the PRRTE term for an allocatable unit where we
can launch a process. Thus, the number of slots equates to the
maximum number of processes PRRTE may start on that node without
oversubscribing it.

23
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-debug-daemons-file.rst Normal file
View File

@@ -0,0 +1,23 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Debug daemon output is enabled and all output from the daemons is
redirected into files with names of the form:
.. code::
output-prted-<daemon-nspace>-<nodename>.log
These names avoid conflict on shared file systems. The files are
located in the top-level session directory assigned to the DVM.

17
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-debug-daemons.rst Normal file
View File

@@ -0,0 +1,17 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Debug daemon output enabled. This is a somewhat limited stream of
information normally used to simply confirm that the daemons
started. Includes leaving the output streams open.

53
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-display.rst Normal file
View File

@@ -0,0 +1,53 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
The ``display`` command line directive must be accompanied by a
comma-delimited list of case-insensitive options indicating what
information about the job and/or allocation is to be displayed. The
full directive need not be provided |mdash| only enough characters are
required to uniquely identify the directive. For example, ``ALL`` is
sufficient to represent the ``ALLOCATION`` directive |mdash| while ``MAP``
can not be used to represent ``MAP-DEVEL`` (though ``MAP-D`` would
suffice).
Supported values include:
* ``ALLOCATION`` displays the detected hosts and slot assignments for
this job
* ``BINDINGS`` displays the resulting bindings applied to processes in
this job
* ``MAP`` displays the resulting locations assigned to processes in
this job
* ``MAP-DEVEL`` displays a more detailed report on the locations
assigned to processes in this job that includes local and node
ranks, assigned bindings, and other data
* ``TOPO=LIST`` displays the topology of each node in the
semicolon-delimited list that is allocated to the job
* ``CPUS[=LIST]`` displays the available CPUs on the provided
semicolon-delimited list of nodes (defaults to all nodes)
The display command line directive can include qualifiers by adding a
colon (``:``) and any combination of one or more of the following
(delimited by colons):
* ``PARSEABLE`` directs that the output be provided in a format that
is easily parsed by machines. Note that ``PARSABLE`` is also accepted as
a typical spelling for the qualifier.
Provided qualifiers will apply to *all* of the display directives.

90
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-dvm-hostfile.rst Normal file
View File

@@ -0,0 +1,90 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
PRRTE supports several levels of user-specified host lists based on an
established precedence order. Users can specify a default hostfile
that contains a list of nodes to be used by the DVM. Only one default
hostfile can be provided for a given DVM. In addition, users can
specify a hostfile that contains a list of nodes to be used for a DVM,
or can provide a comma-delimited list of nodes to be used for that DVM
via the ``--host`` command line option.
The precedence order applied to these various options depends to some
extent on the local environment. The following table illustrates how
host and hostfile directives work together to define the set of hosts
upon which a DVM will execute in the absence of a resource manager
(RM):
.. list-table::
:header-rows: 1
:widths: 10 7 10 32
* - Default hostfile
- host
- hostfile
- Result
* - unset
- unset
- unset
- | The DVN will consist solely of the
| local host where the DVM
| was started.
* - unset
- set
- unset
- | Host option defines resource list for the DVM.
* - unset
- unset
- set
- | Hostfile option defines resource list for the DVM.
* - unset
- set
- set
- | Hostfile option defines resource list for the DVM,
| then host filters the list to define the final
| set of nodes to be used by the DVM
* - set
- unset
- unset
- | Default hostfile defines resource list for the DVM
* - set
- set
- unset
- | Default hostfile defines resource list for the DVM,
| then host filters the list to define the final
| set of nodes to be used by the DVM
* - set
- set
- set
- | Default hostfile defines resource list for the DVM,
| then hostfile filters the list, and then host filters
| the list to define the final set of nodes to be
| used by the DVM
This changes somewhat in the presence of an RM as that entity
specifies the initial allocation of nodes. In this case, the default
hostfile, hostfile and host directives are all used to filter the RM's
specification so that a user can utilize different portions of the
allocation for different DVMs. This is done according to the same
precedence order as in the prior table, with the RM providing the
initial pool of nodes.
.. include:: /prrte-rst-content/detail-hostfiles.rst

59
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-dvm.rst Normal file
View File

@@ -0,0 +1,59 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
A required argument is passed to the ``--dvm`` directive to specify
the location of the DVM controller (e.g., ``--dvm pid:12345``) or by
passing the string ``search`` to instead search for an existing
controller.
Supported options include:
* ``search``: directs the tool to search for available DVM controllers
it is authorized to use, connecting to the first such candidate it
finds.
* ``pid:<arg>``: provides the PID of the target DVM controller. This
can be given as either the PID itself (arg = int) or the path to a
file that contains the PID (arg = ``file:<path>``)
* ``file:<path>``: provides the path to a PMIx rendezvous file that is
output by PMIx servers |mdash| the file contains all the required
information for completing the connection
* ``uri:<arg>``: specifies the URI of the DVM controller, or the name of
the file (specified as ``file:filename``) that contains that info
* ``ns:<arg>``: specifies the namespace of the DVM controller
* ``system``: exclusively find and use the system-level DVM controller
* ``system-first``: look for a system-level DVM controller, fall back
to searching for an available DVM controller the command is
authorized to use if a system-level controller is not found
Examples:
.. code::
prterun --dvm file:dvm_uri.txt --np 4 ./a.out
prterun --dvm pid:12345 --np 4 ./a.out
prterun --dvm uri:file:dvm_uri.txt --np 4 ./a.out
prterun --dvm ns:prte-node1-2095 --np 4 ./a.out
prterun --dvm pid:file:prte_pid.txt --np 4 ./a.out
prterun --dvm search --np 4 ./a.out

18
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-forward-signals.rst Normal file
View File

@@ -0,0 +1,18 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Comma-delimited list of additional signals (names or integers) to
forward to application processes (``none`` = forward
nothing). Signals provided by default include SIGTSTP, SIGUSR1,
SIGUSR2, SIGABRT, SIGALRM, and SIGCONT.

56
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-launcher-hostfile.rst Normal file
View File

@@ -0,0 +1,56 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
PRRTE supports several levels of user-specified hostfiles based on an
established precedence order. Users can specify a hostfile that
contains a list of nodes to be used for the job, or can provide a
comma-delimited list of nodes to be used for that job via the
``--host`` command line option.
The precedence order applied to these various options depends to some
extent on the local environment. The following table illustrates how
host and hostfile directives work together to define the set of hosts
upon which a DVM will execute the job in the absence of a resource
manager (RM):
.. list-table::
:header-rows: 1
:widths: 7 10 45
* - host
- hostfile
- Result
* - unset
- unset
- | The DVM will utilize all its available resources
| when mapping the job.
* - set
- unset
- | Host option defines resource list for the job
* - unset
- set
- | Hostfile defines resource list for the job
* - set
- set
- | Hostfile defines resource list for the job,
| then host filters the list to define the final
| set of nodes to be used for the job
.. include:: /prrte-rst-content/detail-hostfiles.rst
.. include:: /prrte-rst-content/detail-hosts-relative-indexing.rst

19
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-leave-session-attached.rst Normal file
View File

@@ -0,0 +1,19 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Do not discard stdout/stderr of remote PRRTE daemons. The primary use
for this option is to ensure that the daemon output streams (i.e.,
stdout and stderr) remain open after launch, thus allowing the user to
see any daemon-generated error messages. Otherwise, the daemon will
"daemonize" itself upon launch, thereby closing its output streams.

139
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-map-by.rst Normal file
View File

@@ -0,0 +1,139 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Processes are mapped based on one of the following directives as
applied at the job level:
* ``SLOT`` assigns procs to each node up to the number of available
slots on that node before moving to the next node in the
allocation
* ``HWTHREAD`` assigns a proc to each hardware thread on a node in a
round-robin manner up to the number of available slots on that
node before moving to the next node in the allocation
* ``CORE`` (default) assigns a proc to each core on a node in a
round-robin manner up to the number of available slots on that
node before moving to the next node in the allocation
* ``L1CACHE`` assigns a proc to each L1 cache on a node in a
round-robin manner up to the number of available slots on that
node before moving to the next node in the allocation
* ``L2CACHE`` assigns a proc to each L2 cache on a node in a
round-robin manner up to the number of available slots on that
node before moving to the next node in the allocation
* ``L3CACHE`` assigns a proc to each L3 cache on a node in a
round-robin manner up to the number of available slots on that
node before moving to the next node in the allocation
* ``NUMA`` assigns a proc to each NUMA region on a node in a
round-robin manner up to the number of available slots on that
node before moving to the next node in the allocation
* ``PACKAGE`` assigns a proc to each package on a node in a
round-robin manner up to the number of available slots on that
node before moving to the next node in the allocation
* ``NODE`` assigns processes in a round-robin fashion to all nodes
in the allocation, with the number assigned to each node capped
by the number of available slots on that node
* ``SEQ`` (often accompanied by the file=<path> qualifier) assigns
one process to each node specified in the file. The sequential
file is to contain an entry for each desired process, one per
line of the file.
* ``PPR:N``:resource maps N procs to each instance of the specified
resource type in the allocation
* ``RANKFILE`` (often accompanied by the file=<path> qualifier) assigns
one process to the node/resource specified in each entry of the
file, one per line of the file.
* ``PE-LIST=a,b`` assigns procs to each node in the allocation based on
the ORDERED qualifier. The list is comprised of comma-delimited
ranges of CPUs to use for this job. If the ORDERED qualifier is not
provided, then each node will be assigned procs up to the number of
available slots, capped by the availability of the specified CPUs.
If ORDERED is given, then one proc will be assigned to each of the
specified CPUs, if available, capped by the number of slots on each
node and the total number of specified processes. Providing the
OVERLOAD qualifier to the "bind-to" option removes the check on
availability of the CPU in both cases.
Any directive can include qualifiers by adding a colon (``:``) and any
combination of one or more of the following (delimited by colons) to
the ``--map-by`` option (except where noted):
* ``PE=n`` bind n CPUs to each process (can not be used in combination
with rankfile or pe-list directives)
* ``SPAN`` load balance the processes across the allocation by treating
the allocation as a single "super-node" (can not be used in
combination with ``slot``, ``node``, ``seq``, ``ppr``, ``rankfile``, or
``pe-list`` directives)
* ``OVERSUBSCRIBE`` allow more processes on a node than processing elements
* ``NOOVERSUBSCRIBE`` means ``!OVERSUBSCRIBE``
* ``NOLOCAL`` do not launch processes on the same node as ``prun``
* ``HWTCPUS`` use hardware threads as CPU slots
* ``CORECPUS`` use cores as CPU slots (default)
* ``INHERIT`` indicates that a child job (i.e., one spawned from within
an application) shall inherit the placement policies of the parent job
that spawned it.
* ``NOINHERIT`` means ```!INHERIT``
* ``FILE=<path>`` (path to file containing sequential or rankfile entries).
* ``ORDERED`` only applies to the ``PE-LIST`` option to indicate that
procs are to be bound to each of the specified CPUs in the order in
which they are assigned (i.e., the first proc on a node shall be
bound to the first CPU in the list, the second proc shall be bound
to the second CPU, etc.)
.. note:: Directives and qualifiers are case-insensitive and can be
shortened to the minimum number of characters to uniquely
identify them. Thus, ``L1CACHE`` can be given as ``l1cache`` or
simply as ``L1``.
The type of CPU (core vs hwthread) used in the mapping algorithm
is determined as follows:
* by user directive on the command line via the HWTCPUS qualifier to
the ``--map-by`` directive
* by setting the ``rmaps_default_mapping_policy`` MCA parameter to
include the ``HWTCPUS`` qualifier. This parameter sets the default
value for a PRRTE DVM |mdash| qualifiers are carried across to DVM jobs
started via ``prun`` unless overridden by the user's command line
* defaults to CORE in topologies where core CPUs are defined, and to
hwthreads otherwise.
If your application uses threads, then you probably want to ensure that
you are either not bound at all (by specifying ``--bind-to none``), or
bound to multiple cores using an appropriate binding level or specific
number of processing elements per application process via the ``PE=#``
qualifier to the ``--map-by`` command line directive.
A more detailed description of the mapping, ranking, and binding
procedure can be obtained via the ``--help placement`` option.

19
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-noprefix.rst Normal file
View File

@@ -0,0 +1,19 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Disable automatic ``--prefix`` behavior. PRRTE automatically sets the
prefix for remote daemons if it was either configured with the
``--enable-prte-prefix-by-default`` option OR prte itself was executed
with an absolute path to the ``prte`` command. This option disables
that behavior.

55
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-output.rst Normal file
View File

@@ -0,0 +1,55 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
The ``output`` command line directive must be accompanied by a
comma-delimited list of case-insensitive options that control how
output is generated. The full directive need not be provided |mdash| only
enough characters are required to uniquely identify the directive. For
example, ``MERGE`` is sufficient to represent the
``MERGE-STDERR-TO-STDOUT`` directive |mdash| while ``TAG`` can not be
used to represent ``TAG-DETAILED`` (though ``TAG-D`` would suffice).
Supported values include:
* ``TAG`` marks each output line with the ``[job,rank]<stream>:`` of
the process that generated it
* ``TAG-DETAILED`` marks each output line with a detailed annotation
containing ``[namespace,rank][hostname:pid]<stream>:`` of the
process that generated it
* ``TAG-FULLNAME`` marks each output line with the
``[namespace,rank]<stream>:`` of the process that generated it
* ``TAG-FULLNAME`` marks each output line with the
``[namespace,rank]<stream>:`` of the process that generated it
* ``TIMESTAMP`` prefixes each output line with a ``[datetime]<stream>:``
stamp. Note that the timestamp will be the time when the line is
output by the DVM and not the time when the source output it
* ``XML`` provides all output in a pseudo-XML format
``MERGE-STDERR-TO-STDOUT`` merges stderr into stdout
* ``DIR=DIRNAME`` redirects output from application processes into
``DIRNAME/job/rank/std[out,err,diag]``. The provided name will be
converted to an absolute path
* ``FILE=FILENAME`` redirects output from application processes into
``filename.rank.`` The provided name will be converted to an absolute
path
Supported qualifiers include ``NOCOPY`` (do not copy the output to the
stdout/err streams), and ``RAW`` (do not buffer the output into complete
lines, but instead output it as it is received).

21
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-personality.rst Normal file
View File

@@ -0,0 +1,21 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Specify the personality to be used. This governs selection of the
plugin responsible for defining and parsing the command line,
harvesting and forwarding environmental variables, and providing
library-dependent support to the launched processes. Examples include
``ompi`` for an application compiled with Open MPI, ``mpich`` for one
built against the MPICH library, or ``oshmem`` for an OpenSHMEM
application compiled against SUNY's reference library.

18
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-pmixmca.rst Normal file
View File

@@ -0,0 +1,18 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Pass a PMIx MCA parameter
Syntax: ``--pmixmca <key> <value>``, where ``key`` is the parameter
name and ``value`` is the parameter value.

20
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-prefix.rst Normal file
View File

@@ -0,0 +1,20 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Prefix to be used to look for PRRTE executables. PRRTE automatically
sets the prefix for remote daemons if it was either configured with
the ``--enable-prte-prefix-by-default`` option OR prte itself was
executed with an absolute path to the prte command. This option
overrides those settings, if present, and forces use of the provided
path.

18
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-prtemca.rst Normal file
View File

@@ -0,0 +1,18 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Pass a PRRTE MCA parameter.
Syntax: ``--prtemca <key> <value>``, where ``key`` is the parameter
name and ``value`` is the parameter value.

66
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-rank-by.rst Normal file
View File

@@ -0,0 +1,66 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
PRRTE automatically ranks processes for each job starting from zero.
Regardless of the algorithm used, rank assignments span applications
in the same job |mdash| i.e., a command line of
.. code::
-n 3 app1 : -n 2 app2
will result in ``app1`` having three processes ranked 0-2 and ``app2``
having two processes ranked 3-4.
By default, process ranks are assigned in accordance with the mapping
directive |mdash| e.g., jobs that are mapped by-node will have the process
ranks assigned round-robin on a per-node basis. However, users can override
the default by specifying any of the following directives using the
``--rank-by`` command line option:
* ``SLOT`` assigns ranks to each process on a node in the order in
which the mapper assigned them. This is the default behavior,
but is provided as an explicit option to allow users to override
any alternative default specified in the environment. When mapping
to a specific resource type, procs assigned to a given instance
of that resource on a node will be ranked on a per-resource basis
on that node before moving to the next node.
* ``NODE`` assigns ranks round-robin on a per-node basis
* ``FILL`` assigns ranks to procs mapped to a particular resource type
on each node, filling all ranks on that resource before moving to
the next resource on that node. For example, procs mapped by
``L1cache`` would have all procs on the first ``L1cache`` ranked
sequentially before moving to the second ``L1cache`` on the
node. Once all procs on the node have been ranked, ranking would
continue on the next node.
* ``SPAN`` assigns ranks round-robin to procs mapped to a particular
resource type, treating the collection of resource instances
spanning the entire allocation as a single "super node" before
looping around for the next pass. Thus, ranking would begin with the
first proc on the first ``L1cache`` on the first node, then the next
rank would be assigned to the first proc on the second ``L1cache``
on that node, proceeding across until the first proc had been ranked
on all ``L1cache`` used by the job before circling around to rank
the second proc on each object.
The ``rank-by`` command line option has no qualifiers.
.. note:: Directives are case-insensitive. ``SPAN`` is the same as
``span``.
A more detailed description of the mapping, ranking, and binding
procedure can be obtained via the ``--help placement`` option.

167
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-runtime-options.rst Normal file
View File

@@ -0,0 +1,167 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
The ``--runtime-options`` command line directive must be accompanied
by a comma-delimited list of case-insensitive options that control the
runtime behavior of the job. The full directive need not be provided
|mdash| only enough characters are required to uniquely identify the
directive.
Runtime options are typically ``true`` or ``false``, though this is
not a requirement on developers. Since the value of each option may
need to be set (e.g., to override a default set by MCA parameter), the
syntax of the command line directive includes the use of an ``=``
character to allow inclusion of a value for the option. For example,
one can set the ``ABORT-NONZERO-STATUS`` option to ``true`` by
specifying it as ``ABORT-NONZERO-STATUS=1``. Note that boolean options
can be set to ``true`` using a non-zero integer or a case-insensitive
string of the word ``true``. For the latter representation, the user
need only provide at least the ``T`` character. The same policy
applies to setting a boolean option to ``false``.
Note that a boolean option will default to ``true`` if provided
without a value. Thus, ``--runtime-options abort-nonzero`` is
sufficient to set the ``ABORT-NONZERO-STATUS`` option to ``true``.
Supported values include:
* ``ERROR-NONZERO-STATUS[=(bool)]``: if set to false, this directs the
runtime to treat a process that exits with non-zero status as a
normal termination. If set to true, the runtime will consider such
an occurrence as an error termination and take appropriate action
|mdash| i.e., the job will be terminated unless a runtime option
directs otherwise. This option defaults to a true value if the
option is given without a value.
* ``DONOTLAUNCH``: directs the runtime to map but not launch the
specified job. This is provided to help explore possible process
placement patterns before actually starting execution. No value need
be passed as this is not an option that can be set by default in
PRRTE.
* ``SHOW-PROGRESS[=(bool)]``: requests that the runtime provide
progress reports on its startup procedure |mdash| i.e., the launch
of its daemons in support of a job. This is typically used to debug
DVM startup on large systems. This option defaults to a true value
if the option is given without a value.
* ``NOTIFYERRORS[=(bool)]``: if set to true, requests that the runtime
provide a PMIx event whenever a job encounters an error |mdash|
e.g., a process fails. The event is to be delivered to each
remaining process in the job. This option defaults to a true value
if the option is given without a value. See ``--help
notifications`` for more detail as to the PMIx event codes available
for capturing failure events.
* ``RECOVERABLE[=(bool)]``: if set to true, this indicates that the
application wishes to consider the job as recoverable |mdash| i.e.,
the application is assuming responsibility for recovering from any
process failure. This could include application-driven spawn of a
substitute process or internal compensation for the missing
process. This option defaults to a true value if the option is given
without a value.
* ``AUTORESTART[=(bool)]``: if set to true, this requests that the
runtime automatically restart failed processes up to "max restarts"
number of times. This option defaults to a true value if the option
is given without a value.
* ``CONTINUOUS[=(bool)]``: if set to true, this informs the runtime
that the processes in this job are to run until explicitly
terminated. Processes that fail are to be automatically restarted up
to "max restarts" number of times. Notification of process failure
is to be delivered to all processes in the application. This is the
equivalent of specifying ``RECOVERABLE``, ``NOTIFYERRORS``, and
``AUTORESTART`` options except that the runtime, not the
application, assumes responsibility for process recovery. This
option defaults to a true value if the option is given without a
value.
* ``MAX-RESTARTS=<int>``: indicates the maximum number of times a
given process is to be restarted. This can be set at the application
or job level (which will then apply to all applications in that
job).
* ``EXEC-AGENT=<path>`` indicates the executable that shall be used to
start an application process. The resulting command for starting an
application process will be ``<path> app <app-argv>``. The path may
contain its own command line arguments.
* ``DEFAULT-EXEC-AGENT``: directs the runtime to use the system
default exec agent to start an application process. No value need be
passed as this is not an option that can be set by default in PRRTE.
* ``OUTPUT-PROCTABLE[(=channel)]``: directs the runtime to report the
convential debugger process table (includes PID and host location of
each process in the application). Output is directed to stdout if
the channel is ``-``, stderr if ``+``, or into the specified file
otherwise. If no channel is specified, output will be directed to
stdout.
* ``STOP-ON-EXEC``: directs the runtime to stop the application
process(es) immediately upon exec'ing them. The directive will apply
to all processes in the job.
* ``STOP-IN-INIT``: indicates that the runtime should direct the
application process(es) to stop in ``PMIx_Init()``. The directive
will apply to all processes in the job.
* ``STOP-IN-APP``: indicates that the runtime should direct
application processes to stop at some application-defined place and
notify they are ready-to-debug. The directive will apply to all
processes in the job.
* ``TIMEOUT=<string>``: directs the runtime to terminate the job after
it has executed for the specified time. Time is specified in
colon-delimited format |mdash| e.g., ``01:20:13:05`` to indicate 1
day, 20 hours, 13 minutes and 5 seconds. Time specified without
colons will be assumed to have been given in seconds.
* ``SPAWN-TIMEOUT=<string>``: directs the runtime to terminate the job
if job launch is not completed within the specified time. Time is
specified in colon-delimited format |mdash| e.g., ``01:20:13:05`` to
indicate 1 day, 20 hours, 13 minutes and 5 seconds. Time specified
without colons will be assumed to have been given in seconds.
* ``REPORT-STATE-ON-TIMEOUT[(=bool)]``: directs the runtime to provide
a detailed report on job and application process state upon job
timeout. This option defaults to a true value if the option is given
without a value.
* ``GET-STACK-TRACES[(=bool)]``: requests that the runtime provide
stack traces on all application processes still executing upon
timeout. This option defaults to a true value if the option is given
without a value.
* ``REPORT-CHILD-JOBS-SEPARATELY[(=bool)]``: directs the runtime to
report the exit status of any child jobs spawned by the primary job
separately. If false, then the final exit status reported will be
zero if the primary job and all spawned jobs exit normally, or the
first non-zero status returned by either primary or child jobs.
This option defaults to a true value if the option is given without
a value.
* ``AGGREGATE-HELP-MESSAGES[(=bool)]``: directs the runtime to
aggregate help messages, reporting each unique help message once
accompanied by the number of processes that reported it. This option
defaults to a true value if the option is given without a value.
* ``FWD-ENVIRONMENT[(=bool)]``: directs the runtime to forward the
entire local environment in support of the application. This option
defaults to a true value if the option is given without a value.
The ``--runtime-options`` command line option has no qualifiers.
.. note:: Directives are case-insensitive. ``FWD-ENVIRONMENT`` is the
same as ``fwd-environment``.

19
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-stream-buffering.rst Normal file
View File

@@ -0,0 +1,19 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Adjust buffering for stdout/stderr. Allowable values:
* 0: unbuffered
* 1: line buffered
* 2: fully buffered

28
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-tune.rst Normal file
View File

@@ -0,0 +1,28 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Comma-delimited list of one or more files containing PRRTE and PMIx
MCA params for tuning DVM and/or application operations. Parameters in
the file will be treated as *generic* parameters and subject to the
translation rules/uncertainties. See ``--help mca`` for more
information.
Syntax in the file is:
.. code::
param = value
with one parameter and its associated value per line. Empty lines and
lines beginning with the ``#`` character are ignored.

23
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/cli-x.rst Normal file
View File

@@ -0,0 +1,23 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Export an environment variable, optionally specifying a value. For
example:
* ``-x foo`` exports the environment variable ``foo`` and takes its
value from the current environment.
* ``-x foo=bar`` exports the environment variable name ``foo`` and
sets its value to ``bar`` in the started processes.
* ``-x foo*`` exports all current environmental variables starting
with ``foo``.

21
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/definitions-pes.rst Normal file
View File

@@ -0,0 +1,21 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Definition of "processor element"
=================================
By default, PRRTE defines that a "processing element" is a processor
core. However, if ``--use-hwthread-cpus`` is specified on the command
line, then a "processing element" is a hardware thread.

97
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/definitions-slots.rst Normal file
View File

@@ -0,0 +1,97 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Definition of 'slot'
====================
The term "slot" is used extensively in the rest of this documentation.
A slot is an allocation unit for a process. The number of slots on a
node indicate how many processes can potentially execute on that node.
By default, PRRTE will allow one process per slot.
If PRRTE is not explicitly told how many slots are available on a node
(e.g., if a hostfile is used and the number of slots is not specified
for a given node), it will determine a maximum number of slots for
that node in one of two ways:
#. Default behavior: By default, PRRTE will attempt to discover the
number of processor cores on the node, and use that as the number
of slots available.
#. When ``--use-hwthread-cpus`` is used: If ``--use-hwthread-cpus`` is
specified on the command line, then PRRTE will attempt to discover
the number of hardware threads on the node, and use that as the
number of slots available.
This default behavior also occurs when specifying the ``--host``
option with a single host. Thus, the command:
.. code:: sh
shell$ prun --host node1 ./a.out
launches a number of processes equal to the number of cores on node
``node1``, whereas:
.. code:: sh
shell$ prun --host node1 --use-hwthread-cpus ./a.out
launches a number of processes equal to the number of hardware
threads on ``node1``.
When PRRTE applications are invoked in an environment managed by a
resource manager (e.g., inside of a Slurm job), and PRRTE was built
with appropriate support for that resource manager, then PRRTE will
be informed of the number of slots for each node by the resource
manager. For example:
.. code:: sh
shell$ prun ./a.out
launches one process for every slot (on every node) as dictated by
the resource manager job specification.
Also note that the one-process-per-slot restriction can be overridden
in unmanaged environments (e.g., when using hostfiles without a
resource manager) if oversubscription is enabled (by default, it is
disabled). Most parallel applications and HPC environments do not
oversubscribe; for simplicity, the majority of this documentation
assumes that oversubscription is not enabled.
Slots are not hardware resources
--------------------------------
Slots are frequently incorrectly conflated with hardware resources.
It is important to realize that slots are an entirely different metric
than the number (and type) of hardware resources available.
Here are some examples that may help illustrate the difference:
#. More processor cores than slots: Consider a resource manager job
environment that tells PRRTE that there is a single node with 20
processor cores and 2 slots available. By default, PRRTE will
only let you run up to 2 processes.
Meaning: you run out of slots long before you run out of processor
cores.
#. More slots than processor cores: Consider a hostfile with a single
node listed with a ``slots=50`` qualification. The node has 20
processor cores. By default, PRRTE will let you run up to 50
processes.
Meaning: you can run many more processes than you have processor
cores.

20
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-bind-to-core.rst Normal file
View File

@@ -0,0 +1,20 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Bind each process to its own core.
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--bind-to core``.

20
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-display-allocation.rst Normal file
View File

@@ -0,0 +1,20 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Display the allocation being used by this job.
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--display alloc``.

21
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-display-devel-allocation.rst Normal file
View File

@@ -0,0 +1,21 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Display a detailed list (mostly intended for developers) of the
allocation being used by this job.
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--display alloc-devel``.

21
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-display-devel-map.rst Normal file
View File

@@ -0,0 +1,21 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Display a detailed process map (mostly intended for developers)
just before launch.
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--display map-devel``.

20
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-display-map.rst Normal file
View File

@@ -0,0 +1,20 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Display the process map just before launch.
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--display map``.

21
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-display-topo.rst Normal file
View File

@@ -0,0 +1,21 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Display the topology as part of the process map (mostly intended
for developers) just before launch.
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--display topo``.

31
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-gmca.rst Normal file
View File

@@ -0,0 +1,31 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Syntax: ``--gmca <key> <value>``, where ``key`` is the parameter name
and ``value`` is the parameter value. The ``g`` prefix indicates that
this parameter is "global", and to be applied to *all* application
contexts |mdash| not just the one in which the directive appears.
Pass generic MCA parameters |mdash| i.e., parameters whose project
affiliation must be determined by PRRTE based on matching the name of
the parameter with defined values from various projects that PRRTE
knows about.
.. admonition:: Deprecated
:class: warning
This translation can be incomplete (e.g., if known project adds or
changes parameters) |mdash| thus, it is strongly recommended that
users use project-specific parameters such as ``--gprtemca`` or
``--gpmixmca``.

29
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-mca.rst Normal file
View File

@@ -0,0 +1,29 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Syntax: ``--mca <key> <value>``, where ``key`` is the parameter name
and ``value`` is the parameter value.
Pass generic MCA parameters |mdash| i.e., parameters whose project
affiliation must be determined by PRRTE based on matching the name of
the parameter with defined values from various projects that PRRTE
knows about.
.. admonition:: Deprecated
:class: warning
This translation can be incomplete (e.g., if a project adds or
changes parameters) |mdash| thus, it is strongly recommended that
users use project-specific parameters such as ``--prtemca`` or
``--pmixmca``.

20
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-merge-stderr-to-stdout.rst Normal file
View File

@@ -0,0 +1,20 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Merge stderr to stdout for each process.
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--output merge``

26
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-output-directory.rst Normal file
View File

@@ -0,0 +1,26 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Redirect output from application processes into
``filename/job/rank/std[out,err,diag]``. A relative path value will be
converted to an absolute path. The directory name may include a colon
followed by a comma-delimited list of optional case-insensitive
directives. Supported directives currently include ``NOJOBID`` (do not
include a job-id directory level) and ``NOCOPY`` (do not copy the
output to the stdout/err streams).
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--output dir=<path>``.

25
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-output-filename.rst Normal file
View File

@@ -0,0 +1,25 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Redirect output from application processes into ``filename.rank``. A
relative path value will be converted to an absolute path. The
directory name may include a colon followed by a comma-delimited list
of optional case-insensitive directives. Supported directives
currently include ``NOCOPY`` (do not copy the output to the stdout/err
streams).
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--output file=<path>``

20
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-report-bindings.rst Normal file
View File

@@ -0,0 +1,20 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Display process bindings to stderr.
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--display bindings``.

20
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-tag-output.rst Normal file
View File

@@ -0,0 +1,20 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Tag all output with ``[job,rank]``.
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--output``.

20
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-timestamp-output.rst Normal file
View File

@@ -0,0 +1,20 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Timestamp all application process output.
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--output timestamp``.

20
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/deprecated-xml.rst Normal file
View File

@@ -0,0 +1,20 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Provide all output in XML format.
.. admonition:: Deprecated
:class: warning
This option is deprecated. Please use ``--output``.

48
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/detail-hostfiles.rst Normal file
View File

@@ -0,0 +1,48 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Hostfiles (sometimes called "machine files") are a combination of two
things:
#. A listing of hosts on which to launch processes.
#. Optionally, limit the number of processes which can be launched on
each host.
Hostfile syntax consists of one node name on each line, optionally
including a designated number of "slots":
.. code:: sh
# This is a comment line, and will be ignored
node01 slots=10
node13 slots=5
node15
node16
node17 slots=3
...
Blank lines and lines beginning with a ``#`` are ignored.
A "slot" is the PRRTE term for an allocatable unit where we can launch
a process. See the section on definition of the term ``slot`` for a
longer description of slots.
In the absence of the ``slot`` parameter, PRRTE will assign either the
number of slots to be the number of CPUs detected on the node or the
resource manager-assigned value if operating in the presence of an
RM.
.. important:: If using a resource manager, the user-specified number
of slots is capped by the RM-assigned value.

47
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/detail-hosts-cli.rst Normal file
View File

@@ -0,0 +1,47 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Listing Hosts on the Command Line
=================================
Many PRRTE commands accept the ``--host`` CLI parameter.
``--host`` accepts a comma-delimited list of tokens of the form:
.. code::
host[:slots]
The ``host`` token can be either:
* A name that resolves to an IP address, or
* An IP address
.. note:: The names and/or IP addresses of hosts are *only* used for
identifying the target host on which to launch. They are
*not* used for determining which network interfaces are used
by applications (e.g., MPI or other network-based
applications).
For network-based applications, consult their documentation
for how to specify which network interfaces are used.
The optional integer ``:slots`` parameter tells PRRTE the maximum
number of slots to use on that host (see this section on definition
of the term ``slot`` for a description of what a "slot" is).
For example:
.. code::
prterun --host node1:10,node2,node3:5 ...

127
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/detail-hosts-relative-indexing.rst Normal file
View File

@@ -0,0 +1,127 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Relative host indexing
======================
Hostfile and ``--host`` specifications can also be made using relative
indexing. This allows a user to stipulate which hosts are to be used
for a given app context without specifying the particular host name,
but rather its relative position in the allocation.
This can probably best be understood through consideration of a few
examples. Consider the case where a DVM is comprised of a set of nodes
named ``foo1``, ``foo2``, ``foo3``, ``foo4``. The user wants the first
app context to have exclusive use of the first two nodes, and a second
app context to use the last two nodes. Of course, the user could
printout the allocation to find the names of the nodes allocated to
them and then use ``--host`` to specify this layout, but this is
cumbersome and would require hand-manipulation for every invocation.
A simpler method is to utilize PRRTE's relative indexing capability to
specify the desired layout. In this case, a command line containing:
.. code::
--host +n1,+n2 ./app1 : --host +n3,+n4 ./app2
would provide the desired pattern. The ``+`` syntax indicates that the
information is being provided as a relative index into the existing
allocation. Two methods of relative indexing are supported:
* ``+n#``: A relative index into the allocation referencing the ``#``
node. PRRTE will substitute the ``#`` node in the allocation
* ``+e[:#]``: A request for ``#`` empty nodes |mdash| i.e., PRRTE is
to substitute this reference with nodes that have not yet been used
by any other app_context. If the ``:#`` is not provided, PRRTE will
substitute the reference with all empty nodes. Note that PRRTE does
track the empty nodes that have been assigned in this manner, so
multiple uses of this option will result in assignment of unique
nodes up to the limit of the available empty nodes. Requests for
more empty nodes than are available will generate an error.
Relative indexing can be combined with absolute naming of hosts in any
arbitrary manner, and can be used in hostfiles as well as with the
``--host`` command line option. In addition, any slot specification
provided in hostfiles will be respected |mdash| thus, a user can
specify that only a certain number of slots from a relative indexed
host are to be used for a given app context.
Another example may help illustrate this point. Consider the case
where the user has a hostfile containing:
.. code::
dummy1 slots=4
dummy2 slots=4
dummy3 slots=4
dummy4 slots=4
dummy5 slots=4
This may, for example, be a hostfile that describes a set of
commonly-used resources that the user wishes to execute applications
against. For this particular application, the user plans to map
byslot, and wants the first two ranks to be on the second node of any
allocation, the next ranks to land on an empty node, have one rank
specifically on ``dummy4``, the next rank to be on the second node of the
allocation again, and finally any remaining ranks to be on whatever
empty nodes are left. To accomplish this, the user provides a hostfile
of:
.. code::
+n2 slots=2
+e:1
dummy4 slots=1
+n2
+e
The user can now use this information in combination with PRRTE's
sequential mapper to obtain their specific layout:
.. code::
<launcher> --hostfile dummyhosts --hostfile mylayout --prtemca rmaps seq ./my_app
which will result in:
.. code::
rank0 being mapped to dummy3
rank1 to dummy1 as the first empty node
rank2 to dummy4
rank3 to dummy3
rank4 to dummy2 and rank5 to dummy5 as the last remaining unused nodes
Note that the sequential mapper ignores the number of slots arguments
as it only maps one rank at a time to each node in the list.
If the default round-robin mapper had been used, then the mapping
would have resulted in:
* ranks 0 and 1 being mapped to dummy3 since two slots were specified
* ranks 2-5 on dummy1 as the first empty node, which has four slots
* rank6 on dummy4 since the hostfile specifies only a single slot from
that node is to be used
* ranks 7 and 8 on dummy3 since only two slots remain available
* ranks 9-12 on dummy2 since it is the next available empty node and
has four slots
* ranks 13-16 on dummy5 since it is the last remaining unused node and
has four slots
Thus, the use of relative indexing can allow for complex mappings to
be ported across allocations, including those obtained from automated
resource managers, without the need for manual manipulation of scripts
and/or command lines.

23
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/detail-hosts-rm.rst Normal file
View File

@@ -0,0 +1,23 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Resource Manager-Provided Hosts
===============================
When launching under a Resource Manager (RM), the RM usually
picks which hosts |mdash| and how many processes can be launched on
each host |mdash| on a per-job basis.
The RM will communicate this information to PRRTE directly; users can
simply omit specifying hosts or numbers of processes.

155
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/detail-placement-deprecated.rst Normal file
View File

@@ -0,0 +1,155 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Deprecated options
==================
These deprecated options will be removed in a future release.
.. list-table::
:header-rows: 1
:widths: 20 20 30
* - Deprecated Option
- Replacement
- Description
* - ``--bind-to-core``
- ``--bind-to core``
- Bind processes to cores
* - ``--bind-to-socket``
- ``--bind-to package``
- Bind processes to processor sockets
* - ``--bycore``
- ``--map-by core``
- Map processes by core
* - ``--bynode``
- ``--map-by node``
- Launch processes one per node, cycling by node in a round-robin
fashion. This spreads processes evenly among nodes and assigns
ranks in a round-robin, "by node" manner.
* - ``--byslot``
- ``--map-by slot``
- Map and rank processes round-robin by slot
* - ``--cpus-per-proc <#perproc>``
- `--map-by <obj>:PE=<#perproc>``
- Bind each process to the specified number of CPUs
* - ``--cpus-per-rank <#perrank>``
- ``--map-by <obj>:PE=<#perrank>``
- Alias for ``--cpus-per-proc``
* - ``--display-allocation``
- ``--display ALLOC``
- Display the detected resource allocation
* - ``-display-devel-map``
- ``--display MAP-DEVEL``
- Display a detailed process map (mostly intended for developers)
just before launch.
* - ``--display-map``
- ``--display MAP``
- Display a table showing the mapped location of each process
prior to launch.
* - ``--display-topo``
- ``--display TOPO``
- Display the topology as part of the process map (mostly
intended for developers) just before launch.
* - ``--do-not-launch``
- ``--map-by :DONOTLAUNCH``
- Perform all necessary operations to prepare to launch the
application, but do not actually launch it (usually used to
test mapping patterns).
* - ``--do-not-resolve``
- ``--map-by :DONOTRESOLVE``
- Do not attempt to resolve interfaces |mdash| usually used to
determine proposed process placement/binding prior to obtaining
an allocation.
* - ``-N <num>``
- ``--map-by prr:<num>:node``
- Launch ``num`` processes per node on all allocated nodes
* - ``--nolocal``
- ``--map-by :NOLOCAL``
- Do not run any copies of the launched application on the same
node as ``prun`` is running. This option will override listing
the ``localhost`` with ``--host`` or any other host-specifying
mechanism.
* - ``--nooversubscribe``
- ``--map-by :NOOVERSUBSCRIBE``
- Do not oversubscribe any nodes; error (without starting any
processes) if the requested number of processes would cause
oversubscription. This option implicitly sets "max_slots" equal
to the "slots" value for each node. (Enabled by default).
* - ``--npernode <#pernode>``
- ``--map-by ppr:<#pernode>:node``
- On each node, launch this many processes
* - ``--npersocket <#persocket>``
- ``--map-by ppr:<#perpackage>:package``
- On each node, launch this many processes times the number of
processor sockets on the node. The ``--npersocket`` option also
turns on the ``--bind-to socket`` option. The term ``socket``
has been globally replaced with ``package``.
* - ``--oversubscribe``
- ``--map-by :OVERSUBSCRIBE``
- Nodes are allowed to be oversubscribed, even on a managed
system, and overloading of processing elements.
* - ``--pernode``
- ``--map-by ppr:1:node``
- On each node, launch one process
* - ``--ppr``
- `--map-by ppr:<list>``
- Comma-separated list of number of processes on a given resource type
[default: ``none``].
* - ``--rankfile <FILENAME>``
- ``--map-by rankfile:FILE=<FILENAME>``
- Use a rankfile for mapping/ranking/binding
* - ``--report-bindings``
- ``--display BINDINGS``
- Report any bindings for launched processes
* - ``--tag-output``
- ``--output TAG``
- Tag all output with ``[job,rank]``
* - ``--timestamp-output``
- ``--output TIMESTAMP``
- Timestamp all application process output
* - ``--use-hwthread-cpus``
- ``--map-by :HWTCPUS``
- Use hardware threads as independent CPUs
* - ``--xml``
- ``--output XML``
- Provide all output in XML format

91
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/detail-placement-diagnostics.rst Normal file
View File

@@ -0,0 +1,91 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Diagnostics
===========
PRRTE provides various diagnostic reports that aid the user in
verifying and tuning the mapping/ranking/binding for a specific job.
The ``:REPORT`` qualifier to the ``--bind-to`` command line option can
be used to report process bindings.
As an example, consider a node with:
* 2 processor packages,
* 4 cores per package, and
* 8 hardware threads per core.
In each of the examples below the binding is reported in a human readable
format.
.. code::
$ prun --np 4 --map-by core --bind-to core:REPORT ./a.out
[node01:103137] MCW rank 0 bound to package[0][core:0]
[node01:103137] MCW rank 1 bound to package[0][core:1]
[node01:103137] MCW rank 2 bound to package[0][core:2]
[node01:103137] MCW rank 3 bound to package[0][core:3]
In the example above, processes are bound to successive cores on the
first package.
.. code::
$ prun --np 4 --map-by package --bind-to package:REPORT ./a.out
[node01:103115] MCW rank 0 bound to package[0][core:0-9]
[node01:103115] MCW rank 1 bound to package[1][core:10-19]
[node01:103115] MCW rank 2 bound to package[0][core:0-9]
[node01:103115] MCW rank 3 bound to package[1][core:10-19]
In the example above, processes are bound to all cores on successive
packages in a round-robin fashion.
.. code::
$ prun --np 4 --map-by package:PE=2 --bind-to core:REPORT ./a.out
[node01:103328] MCW rank 0 bound to package[0][core:0-1]
[node01:103328] MCW rank 1 bound to package[1][core:10-11]
[node01:103328] MCW rank 2 bound to package[0][core:2-3]
[node01:103328] MCW rank 3 bound to package[1][core:12-13]
The example above shows us that 2 cores have been bound per process.
The ``:PE=2`` qualifier states that 2 CPUs underneath the package
(which would be cores in this case) are mapped to each process.
.. code::
$ prun --np 4 --map-by core:PE=2:HWTCPUS --bind-to :REPORT hostname
[node01:103506] MCW rank 0 bound to package[0][hwt:0-1]
[node01:103506] MCW rank 1 bound to package[0][hwt:8-9]
[node01:103506] MCW rank 2 bound to package[0][hwt:16-17]
[node01:103506] MCW rank 3 bound to package[0][hwt:24-25]
The example above shows us that 2 hardware threads have been bound per
process. In this case ``prun`` is directing the DVM to map by
hardware threads since we used the ``:HWTCPUS`` qualifier. Without
that qualifier this command would return an error since by default the
DVM will not map to resources smaller than a core. The ``:PE=2``
qualifier states that 2 processing elements underneath the core (which
would be hardware threads in this case) are mapped to each process.
.. code::
$ prun --np 4 --bind-to none:REPORT hostname
[node01:107126] MCW rank 0 is not bound (or bound to all available processors)
[node01:107126] MCW rank 1 is not bound (or bound to all available processors)
[node01:107126] MCW rank 2 is not bound (or bound to all available processors)
[node01:107126] MCW rank 3 is not bound (or bound to all available processors)
Binding is turned off in the above example, as reported.

434
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/detail-placement-examples.rst Normal file
View File

@@ -0,0 +1,434 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Examples
========
Listed here are the subset of command line options that will be used
in the process mapping/ranking/binding examples below.
Specifying Host Nodes
---------------------
Use one of the following options to specify which hosts (nodes) within
the PRRTE DVM environment to run on.
.. code::
--host <host1,host2,...,hostN>
# or
--host <host1:X,host2:Y,...,hostN:Z>
* List of hosts on which to invoke processes. After each hostname a
colon (``:``) followed by a positive integer can be used to specify
the number of slots on that host (``:X``, ``:Y``, and ``:Z``). The
default is ``1``.
.. code::
--hostfile <hostfile>
* Provide a hostfile to use.
Process Mapping / Ranking / Binding Options
-------------------------------------------
* ``-c #``, ``-n #``, ``--n #``, ``--np <#>``: Run this many copies of
the program on the given nodes. This option indicates that the
specified file is an executable program and not an application
context. If no value is provided for the number of copies to execute
(i.e., neither the ``-np`` nor its synonyms are provided on the
command line), ``prun`` will automatically execute a copy of the
program on each process slot (see below for description of a
"process slot"). This feature, however, can only be used in the SPMD
model and will return an error (without beginning execution of the
application) otherwise.
.. note:: These options specify the number of processes to launch.
None of the options imply a particular binding policy
|mdash| e.g., requesting ``N`` processes for each package
does not imply that the processes will be bound to the
package.
* ``--map-by <object>``: Map to the specified object. Supported
objects include:
* ``slot``
* ``hwthread``
* ``core`` (default)
* ``l1cache``
* ``l2cache``
* ``l3cache``
* ``numa``
* ``package``
* ``node``
* ``seq``
* ``ppr``
* ``rankfile``
* ``pe-list``
Any object can include qualifiers by adding a colon (``:``) and any
colon-delimited combination of one or more of the following to the
``--map-by`` options:
* ``PE=n`` bind ``n`` processing elements to each process (can not
be used in combination with rankfile or pe-list directives)
.. error:: JMS Several of the options below refer to ``pe-list``.
Is this option supposed to be ``PE-LIST=n``, not
``PE=n``?
* ``SPAN`` load balance the processes across the allocation (cannot
be used in combination with ``slot``, ``node``, ``seq``, ``ppr``,
``rankfile``, or ``pe-list`` directives)
* ``OVERSUBSCRIBE`` allow more processes on a node than processing
elements
* ``NOOVERSUBSCRIBE`` means ``!OVERSUBSCRIBE``
* ``NOLOCAL`` do not launch processes on the same node as ``prun``
* ``HWTCPUS`` use hardware threads as CPU slots
* ``CORECPUS`` use cores as CPU slots (default)
* ``INHERIT`` indicates that a child job (i.e., one spawned from
within an application) shall inherit the placement policies of the
parent job that spawned it.
* ``NOINHERIT`` means ``!INHERIT``
* ``FILE=<path>`` (path to file containing sequential or rankfile
entries).
* ``ORDERED`` only applies to the PE-LIST option to indicate that
procs are to be bound to each of the specified CPUs in the order
in which they are assigned (i.e., the first proc on a node shall
be bound to the first CPU in the list, the second proc shall be
bound to the second CPU, etc.)
``ppr`` policy example: ``--map-by ppr:N:<object>`` will launch
``N`` times the number of objects of the specified type on each
node.
.. note:: Directives and qualifiers are case-insensitive and can be
shortened to the minimum number of characters to uniquely
identify them. Thus, ``L1CACHE`` can be given as
``l1cache`` or simply as ``L1``.
* ``--rank-by <object>``: This assigns ranks in round-robin fashion
according to the specified object. The default follows the mapping
pattern. Supported rank-by objects include:
* ``slot``
* ``node``
* ``fill``
* ``span``
There are no qualifiers for the ``--rank-by`` directive.
* ``--bind-to <object>``: This binds processes to the specified
object. See defaults in Quick Summary. Supported bind-to objects
include:
* ``none``
* ``hwthread``
* ``core``
* ``l1cache``
* ``l2cache``
* ``l3cache``
* ``numa``
* ``package``
Any object can include qualifiers by adding a colon (``:``) and any
colon-delimited combination of one or more of the following to the
``--bind-to`` options:
* ``overload-allowed`` allows for binding more than one process in
relation to a CPU
* ``if-supported`` if binding to that object is supported on this
system.
Specifying Host Nodes
---------------------
Host nodes can be identified on the command line with the ``--host``
option or in a hostfile.
For example, assuming no other resource manager or scheduler is
involved:
.. code::
prun --host aa,aa,bb ./a.out
This launches two processes on node ``aa`` and one on ``bb``.
.. code::
prun --host aa ./a.out
This launches one process on node ``aa``.
.. code::
prun --host aa:5 ./a.out
This launches five processes on node ``aa``.
Or, consider the hostfile:
.. code::
$ cat myhostfile
aa slots=2
bb slots=2
cc slots=2
Here, we list both the host names (``aa``, ``bb``, and ``cc``) but
also how many "slots" there are for each. Slots indicate how many
processes can potentially execute on a node. For best performance, the
number of slots may be chosen to be the number of cores on the node or
the number of processor sockets.
If the hostfile does not provide slots information, the PRRTE DVM will
attempt to discover the number of cores (or hwthreads, if the
``:HWTCPUS`` qualifier to the ``--map-by`` option is set) and set the
number of slots to that value.
Examples using the hostfile above with and without the ``--host``
option:
.. code::
prun --hostfile myhostfile ./a.out
This will launch two processes on each of the three nodes.
.. code::
prun --hostfile myhostfile --host aa ./a.out
This will launch two processes, both on node ``aa``.
.. code::
prun --hostfile myhostfile --host dd ./a.out
This will find no hosts to run on and abort with an error. That is, the
specified host ``dd`` is not in the specified hostfile.
When running under resource managers (e.g., SLURM, Torque, etc.), PRTE
will obtain both the hostnames and the number of slots directly from
the resource manger. The behavior of ``--host`` in that environment
will behave the same as if a hostfile was provided (since it is
provided by the resource manager).
Specifying Number of Processes
------------------------------
As we have just seen, the number of processes to run can be set using
the hostfile. Other mechanisms exist.
The number of processes launched can be specified as a multiple of the
number of nodes or processor sockets available. Consider the hostfile
below for the examples that follow.
.. code::
$ cat myhostfile
aa
bb
For example:
.. code::
prun --hostfile myhostfile --map-by ppr:2:package ./a.out
This launches processes 0-3 on node ``aa`` and process 4-7 on node
``bb``, where ``aa`` and ``bb`` are both dual-package nodes. The
``--map-by ppr:2:package`` option also turns on the ``--bind-to
package`` option, which is discussed in a later section.
.. code::
prun --hostfile myhostfile --map-by ppr:2:node ./a.out
This launches processes 0-1 on node ``aa`` and processes 2-3 on node
``bb``.
.. code::
prun --hostfile myhostfile --map-by ppr:1:node ./a.out
This launches one process per host node.
Another alternative is to specify the number of processes with the
``--np`` option. Consider now the hostfile:
.. code::
$ cat myhostfile
aa slots=4
bb slots=4
cc slots=4
With this hostfile:
.. code::
prun --hostfile myhostfile --np 6 ./a.out
This will launch processes 0-3 on node ``aa`` and processes 4-5 on
node ``bb``. The remaining slots in the hostfile will not be used
since the ``-np`` option indicated that only 6 processes should be
launched.
Mapping Processes to Nodes Using Policies
-----------------------------------------
The examples above illustrate the default mapping of process processes
to nodes. This mapping can also be controlled with various
``prun`` / ``prterun`` options that describe mapping policies.
.. code::
$ cat myhostfile
aa slots=4
bb slots=4
cc slots=4
Consider the hostfile above, with ``--np 6``:
.. list-table::
:header-rows: 1
* - Command
- Ranks on ``aa``
- Ranks on ``bb``
- Ranks on ``cc``
* - ``prun``
- 0 1 2 3
- 4 5
-
* - ``prun --map-by node``
- 0 3
- 1 4
- 2 5
* - ``prun --map-by node:NOLOCAL``
-
- 0 2 4
- 1 3 5
The ``--map-by node`` option will load balance the processes across
the available nodes, numbering each process by node in a round-robin
fashion.
The ``:NOLOCAL`` qualifier to ``--map-by`` prevents any processes from
being mapped onto the local host (in this case node ``aa``). While
``prun`` typically consumes few system resources, the ``:NOLOCAL``
qualifier can be helpful for launching very large jobs where ``prun``
may actually need to use noticeable amounts of memory and/or
processing time.
Just as ``--np`` can specify fewer processes than there are slots, it
can also oversubscribe the slots. For example, with the same hostfile:
.. code::
prun --hostfile myhostfile --np 14 ./a.out
This will produce an error since the default ``:NOOVERSUBSCRIBE``
qualifier to ``--map-by`` prevents oversubscription.
To oversubscribe the nodes you can use the ``:OVERSUBSCRIBE``
qualifier to ``--map-by``:
.. code::
prun --hostfile myhostfile --np 14 --map-by :OVERSUBSCRIBE ./a.out
This will launch processes 0-5 on node ``aa``, 6-9 on ``bb``, and
10-13 on ``cc``.
Limits to oversubscription can also be specified in the hostfile
itself with the ``max_slots`` field:
.. code::
$ cat myhostfile
aa slots=4 max_slots=4
bb max_slots=8
cc slots=4
The ``max_slots`` field specifies such a limit. When it does, the
``slots`` value defaults to the limit. Now:
.. code::
prun --hostfile myhostfile --np 14 --map-by :OVERSUBSCRIBE ./a.out
This causes the first 12 processes to be launched as before, but the
remaining two processes will be forced onto node cc. The other two
nodes are protected by the hostfile against oversubscription by this
job.
Using the ``:NOOVERSUBSCRIBE`` qualifier to ``--map-by`` option can be
helpful since the PRTE DVM currently does not get ``max_slots`` values
from the resource manager.
Of course, ``--np`` can also be used with the ``--host`` option. For
example,
.. code::
prun --host aa,bb --np 8 ./a.out
This will produce an error since the default ``:NOOVERSUBSCRIBE``
qualifier to ``--map-by`` prevents oversubscription.
.. code::
prun --host aa,bb --np 8 --map-by :OVERSUBSCRIBE ./a.out
This launches 8 processes. Since only two hosts are specified, after
the first two processes are mapped, one to ``aa`` and one to ``bb``,
the remaining processes oversubscribe the specified hosts evenly.
.. code::
prun --host aa:2,bb:6 --np 8 ./a.out
This launches 8 processes. Processes 0-1 on node ``aa`` since it has 2
slots and processes 2-7 on node ``bb`` since it has 6 slots.
And here is a MIMD example:
.. code::
prun --host aa --np 1 hostname : --host bb,cc --np 2 uptime
This will launch process 0 running ``hostname`` on node ``aa`` and
processes 1 and 2 each running ``uptime`` on nodes ``bb`` and ``cc``,
respectively.

156
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/detail-placement-fundamentals.rst Normal file
View File

@@ -0,0 +1,156 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Fundamentals
============
The mapping of processes to nodes can be defined not just with general
policies but also, if necessary, using arbitrary mappings that cannot
be described by a simple policy. Supported directives, given on the
command line via the ``--map-by`` option, include:
* ``SEQ``: (often accompanied by the ``file=<path>`` qualifier)
assigns one process to each node specified in the file. The
sequential file is to contain an entry for each desired process, one
per line of the file.
* ``RANKFILE``: (often accompanied by the ``file=<path>`` qualifier)
assigns one process to the node/resource specified in each entry of
the file, one per line of the file.
For example, using the hostfile below:
.. code::
$ cat myhostfile
aa slots=4
bb slots=4
cc slots=4
The command below will launch three processes, one on each of nodes
``aa``, ``bb``, and ``cc``, respectively. The slot counts don't
matter; one process is launched per line on whatever node is listed on
the line.
.. code::
$ prun --hostfile myhostfile --map-by seq ./a.out
Impact of the ranking option is best illustrated by considering the
following hostfile and test cases where each node contains two
packages (each package with two cores). Using the ``--map-by
ppr:2:package`` option, we map two processes onto each package and
utilize the ``--rank-by`` option as show below:
.. code::
$ cat myhostfile
aa
bb
.. list-table::
:header-rows: 1
* - Command
- Ranks on ``aa``
- Ranks on ``bb``
* - ``--rank-by core``
- 0 1 ! 2 3
- 4 5 ! 6 7
* - ``--rank-by package``
- 0 2 ! 1 3
- 4 6 ! 5 7
* - ``--rank-by package:SPAN``
- 0 4 ! 1 5
- 2 6 ! 3 7
Ranking by slot provides the identical result as ranking by core in
this case |mdash| a simple progression of ranks across each
node. Ranking by package does a round-robin ranking across packages
within each node until all processes have been assigned a rank, and
then progresses to the next node. Adding the ``:SPAN`` qualifier to
the ranking directive causes the ranking algorithm to treat the entire
allocation as a single entity |mdash| thus, the process ranks are
assigned across all packages before circling back around to the
beginning.
The binding operation restricts the process to a subset of the CPU
resources on the node.
The processors to be used for binding can be identified in terms of
topological groupings |mdash| e.g., binding to an l3cache will bind
each process to all processors within the scope of a single L3 cache
within their assigned location. Thus, if a process is assigned by the
mapper to a certain package, then a ``--bind-to l3cache`` directive
will cause the process to be bound to the processors that share a
single L3 cache within that package.
To help balance loads, the binding directive uses a round-robin method,
binding a process to the first available specified object type within
the object where the process was mapped. For example, consider the case
where a job is mapped to the package level, and then bound to core. Each
package will have multiple cores, so if multiple processes are mapped to
a given package, the binding algorithm will assign each process located
to a package to a unique core in a round-robin manner.
Binding can only be done to the mapped object or to a resource located
within that object.
An object is considered completely consumed when the number of
processes bound to it equals the number of CPUs within it. Unbound
processes are not considered in this computation. Additional
processes cannot be mapped to consumed objects unless the
OVERLOAD qualifier is provided via the "--bind-to" command
line option.
Default process mapping/ranking/binding policies can also be set with MCA
parameters, overridden by the command line options when provided. MCA
parameters can be set on the ``prte`` command line when starting the
DVM (or in the ``prterun`` command line for a single-execution job), but
also in a system or user ``mca-params.conf`` file or as environment
variables, as described in the MCA section below. Some examples include:
.. list-table::
:header-rows: 1
* - ``prun`` option
- MCA parameter key
- Value
* - ``--map-by core``
- ``rmaps_default_mapping_policy``
- ``core``
* - ``--map-by package``
- ``rmaps_default_mapping_policy``
- ``package``
* - ``--rank-by core``
- ``rmaps_default_ranking_policy``
- ``core``
* - ``--bind-to core``
- ``hwloc_default_binding_policy``
- ``core```
* - ``--bind-to package``
- ``hwloc_default_binding_policy``
- ``package``
* - ``--bind-to none``
- ``hwloc_default_binding_policy``
- ``none``

212
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/detail-placement-limits.rst Normal file
View File

@@ -0,0 +1,212 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Overloading and Oversubscribing
===============================
This section explores the difference between the terms "overloading"
and "oversubscribing". Users are often confused by the difference
between these two scenarios. As such, this section provides a number
of scenarios to help illustrate the differences.
* ``--map-by :OVERSUBSCRIBE`` allow more processes on a node than
allocated
* ``--bind-to <object>:overload-allowed`` allows for binding more than
one process in relation to a CPU
The important thing to remember with *oversubscribing* is that it can
be defined separately from the actual number of CPUs on a node. This
allows the mapper to place more or fewer processes per node than
CPUs. By default, PRRTE uses cores to determine slots in the absence
of such information provided in the hostfile or by the resource
manager (except in the case of the ``--host`` as described in
the section on that command line option.
The important thing to remember with *overloading* is that it is
defined as binding more processes than CPUs. By default, PRRTE uses
cores as a means of counting the number of CPUs. However, the user can
adjust this. For example when using the ``:HWTCPUS`` qualifier to the
``--map-by`` option PRRTE will use hardware threads as a means of
counting the number of CPUs.
For the following examples consider a node with:
* 2 processor packages,
* 10 cores per package, and
* 8 hardware threads per core.
Consider the node from above with the hostfile below:
.. code::
$ cat myhostfile
node01 slots=32
node02 slots=32
The ``slots`` token tells PRRTE that it can place up to 32 processes
before *oversubscribing* the node.
If we run the following:
.. code::
prun --np 34 --hostfile myhostfile --map-by core --bind-to core hostname
It will return an error at the binding time indicating an
*overloading* scenario.
The mapping mechanism assigns 32 processes to ``node01`` matching the
``slots`` specification in the hostfile. The binding mechanism will bind
the first 20 processes to unique cores leaving it with 12 processes
that it cannot bind without overloading one of the cores (putting more
than one process on the core).
Using the ``overload-allowed`` qualifier to the ``--bind-to core``
option tells PRRTE that it may assign more than one process to a core.
If we run the following:
.. code::
prun --np 34 --hostfile myhostfile --map-by core --bind-to core:overload-allowed hostname
This will run correctly placing 32 processes on ``node01``, and 2
processes on ``node02``. On ``node01`` two processes are bound to
cores 0-11 accounting for the overloading of those cores.
Alternatively, we could use hardware threads to give binding a lower
level CPU to bind to without overloading.
If we run the following:
.. code::
prun --np 34 --hostfile myhostfile --map-by core:HWTCPUS --bind-to hwthread hostname
This will run correctly placing 32 processes on ``node01``, and 2
processes on ``node02``. On ``node01`` two processes are mapped to
cores 0-11 but bound to different hardware threads on those cores (the
logical first and second hardware thread). Thus no hardware threads
are overloaded at binding time.
In both of the examples above the node is not oversubscribed at
mapping time because the hostfile set the oversubscription limit to
``slots=32`` for each node. It is only after we exceed that limit that
PRRTE will throw an oversubscription error.
Consider next if we ran the following:
.. code::
prun --np 66 --hostfile myhostfile --map-by core:HWTCPUS --bind-to hwthread hostname
This will return an error at mapping time indicating an
oversubscription scenario. The mapping mechanism will assign all of
the available slots (64 across 2 nodes) and be left two processes to
map. The only way to map those processes is to exceed the number of
available slots putting the job into an oversubscription scenario.
You can force PRRTE to oversubscribe the nodes by using the
``:OVERSUBSCRIBE`` qualifier to the ``--map-by`` option as seen in the
example below:
.. code::
prun --np 66 --hostfile myhostfile \
--map-by core:HWTCPUS:OVERSUBSCRIBE --bind-to hwthread hostname
This will run correctly placing 34 processes on ``node01`` and 32 on
``node02``. Each process is bound to a unique hardware thread.
Overloading vs. Oversubscription: Package Example
-------------------------------------------------
Let's extend these examples by considering the package level.
Consider the same node as before, but with the hostfile below:
.. code::
$ cat myhostfile
node01 slots=22
node02 slots=22
The lowest level CPUs are "cores" and we have 20 total (10 per
package).
If we run:
.. code::
prun --np 20 --hostfile myhostfile --map-by package \
--bind-to package:REPORT hostname
Then 10 processes are mapped to each package, and bound at the package
level. This is not overloading since we have 10 CPUs (cores)
available in the package at the hardware level.
However, if we run:
.. code::
prun --np 21 --hostfile myhostfile --map-by package \
--bind-to package:REPORT hostname
Then 11 processes are mapped to the first package and 10 to the second
package. At binding time we have an overloading scenario because
there are only 10 CPUs (cores) available in the package at the
hardware level. So the first package is overloaded.
Overloading vs. Oversubscription: Hardware Threads Example
----------------------------------------------------------
Similarly, if we consider hardware threads.
Consider the same node as before, but with the hostfile below:
.. code::
$ cat myhostfile
node01 slots=165
node02 slots=165
The lowest level CPUs are "hwthreads" (because we are going to use the
``:HWTCPUS`` qualifier) and we have 160 total (80 per package).
If we re-run (from the package example) and add the ``:HWTCPUS``
qualifier:
.. code::
prun --np 21 --hostfile myhostfile --map-by package:HWTCPUS \
--bind-to package:REPORT hostname
Without the ``:HWTCPUS`` qualifier this would be overloading (as we
saw previously). The mapper places 11 processes on the first package
and 10 to the second package. The processes are still bound to the
package level. However, with the ``:HWTCPUS`` qualifier, it is not
overloading since we have 80 CPUs (hwthreads) available in the package
at the hardware level.
Alternatively, if we run:
.. code::
prun --np 161 --hostfile myhostfile --map-by package:HWTCPUS \
--bind-to package:REPORT hostname
Then 81 processes are mapped to the first package and 80 to the second
package. At binding time we have an overloading scenario because
there are only 80 CPUs (hwthreads) available in the package at the
hardware level. So the first package is overloaded.

85
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/detail-placement-rankfiles.rst Normal file
View File

@@ -0,0 +1,85 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Rankfiles
=========
Another way to specify arbitrary mappings is with a rankfile, which
gives you detailed control over process binding as well.
Rankfiles are text files that specify detailed information about how
individual processes should be mapped to nodes, and to which
processor(s) they should be bound. Each line of a rankfile specifies
the location of one process. The general form of each line in the
rankfile is:
.. code::
rank <N>=<hostname> slot=<slot list>
For example:
.. code::
$ cat myrankfile
rank 0=aa slot=10-12
rank 1=bb slot=0,1,4
rank 2=cc slot=1-2
$ prun --host aa,bb,cc,dd --map-by rankfile:FILE=myrankfile ./a.out
Means that:
* Rank 0 runs on node aa, bound to logical cores 10-12.
* Rank 1 runs on node bb, bound to logical cores 0, 1, and 4.
* Rank 2 runs on node cc, bound to logical cores 1 and 2.
Similarly:
.. code::
$ cat myrankfile
rank 0=aa slot=1:0-2
rank 1=bb slot=0:0,1,4
rank 2=cc slot=1-2
$ prun --host aa,bb,cc,dd --map-by rankfile:FILE=myrankfile ./a.out
Means that:
* Rank 0 runs on node aa, bound to logical package 1, cores 10-12 (the
0th through 2nd cores on that package).
* Rank 1 runs on node bb, bound to logical package 0, cores 0, 1,
and 4.
* Rank 2 runs on node cc, bound to logical cores 1 and 2.
The hostnames listed above are "absolute," meaning that actual
resolvable hostnames are specified. However, hostnames can also be
specified as "relative," meaning that they are specified in relation
to an externally-specified list of hostnames (e.g., by ``prun``'s
``--host`` argument, a hostfile, or a job scheduler).
The "relative" specification is of the form "``+n<X>``", where ``X``
is an integer specifying the Xth hostname in the set of all available
hostnames, indexed from 0. For example:
.. code::
$ cat myrankfile
rank 0=+n0 slot=10-12
rank 1=+n1 slot=0,1,4
rank 2=+n2 slot=1-2
$ prun --host aa,bb,cc,dd --map-by rankfile:FILE=myrankfile ./a.out
All package/core slot locations are be specified as *logical*
indexes. You can use tools such as HWLOC's ``lstopo`` to find the
logical indexes of packages and cores.

113
macx64/mpi/openmpi/share/prte/rst/prrte-rst-content/detail-placement.rst Normal file
View File

@@ -0,0 +1,113 @@
.. -*- rst -*-
Copyright (c) 2022-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
.. The following line is included so that Sphinx won't complain
about this file not being directly included in some toctree
Overview
========
PRRTE provides a set of three controls for assigning process
locations and ranks:
#. Mapping: Assigns a default location to each process
#. Ranking: Assigns a unique integer rank value to each process
#. Binding: Constrains each process to run on specific processors
This section provides an overview of these three controls. Unless
otherwise this behavior is shared by ``prun(1)`` (working with a PRRTE
DVM), and ``prterun(1)``. More detail about PRRTE process placement is
available in the following sections (using ``--help
placement-<section>``):
* ``examples``: some examples of the interactions between mapping,
ranking, and binding options.
* ``fundamentals``: provides deeper insight into PRRTE's mapping,
ranking, and binding options.
* ``limits``: explains the difference between *overloading* and
*oversubscribing* resources.
* ``diagnostics``: describes options for obtaining various diagnostic
reports that aid the user in verifying and tuning the placement for
a specific job.
* ``rankfiles``: explains the format and use of the rankfile mapper
for specifying arbitrary process placements.
* ``deprecated``: a list of deprecated options and their new
equivalents.
* ``all``: outputs all the placement help except for the
``deprecated`` section.
Quick Summary
-------------
The two binaries that most influence process layout are ``prte(1)``
and ``prun(1)``. The ``prte(1)`` process discovers the allocation,
establishes a Distributed Virtual Machine by starting a ``prted(1)``
daemon on each node of the allocation, and defines the efault
mapping/ranking/binding policies for all jobs. The ``prun(1)`` process
defines the specific mapping/ranking/binding for a specific job. Most
of the command line controls are targeted to ``prun(1)`` since each job
has its own unique requirements.
``prterun(1)`` is just a wrapper around ``prte(1)`` for a single job
PRRTE DVM. It is doing the job of both ``prte(1)`` and ``prun(1)``,
and, as such, accepts the sum all of their command line arguments. Any
example that uses ``prun(1)`` can substitute the use of ``prterun(1)``
except where otherwise noted.
The ``prte(1)`` process attempts to automatically discover the nodes
in the allocation by querying supported resource managers. If a
supported resource manager is not present then ``prte(1)`` relies on a
hostfile provided by the user. In the absence of such a hostfile it
will run all processes on the localhost.
If running under a supported resource manager, the ``prte(1)`` process
will start the daemon processes (``prted(1)``) on the remote nodes
using the corresponding resource manager process starter. If no such
starter is available then ``ssh`` (or ``rsh``) is used.
Minus user direction, PRRTE will automatically map processes in a
round-robin fashion by CPU, binding each process to its own CPU. The
type of CPU used (core vs hwthread) is determined by (in priority
order):
* user directive on the command line via the HWTCPUS qualifier to
the ``--map-by`` directive
* setting the ``rmaps_default_mapping_policy`` MCA parameter to
include the ``HWTCPUS`` qualifier. This parameter sets the default
value for a PRRTE DVM |mdash| qualifiers are carried across to DVM
jobs started via ``prun`` unless overridden by the user's command
line
* defaulting to ``CORE`` in topologies where core CPUs are defined,
and to ``hwthreads`` otherwise.
By default, the ranks are assigned in accordance with the mapping
directive |mdash| e.g., jobs that are mapped by-node will have the
process ranks assigned round-robin on a per-node basis.
PRRTE automatically binds processes unless directed not to do so by
the user. Minus direction, PRRTE will bind individual processes to
their own CPU within the object to which they were mapped. Should a
node become oversubscribed during the mapping process, and if
oversubscription is allowed, all subsequent processes assigned to that
node will *not* be bound.
.. include:: /prrte-rst-content/definitions-slots.rst
.. include:: /prrte-rst-content/definitions-pes.rst

475
macx64/mpi/openmpi/share/prte/rst/schizo-ompi-rst-content/schizo-ompi-cli.rstxt Normal file
View File

@@ -0,0 +1,475 @@
.. -*- rst -*-
Copyright (c) 2021-2023 Nanook Consulting. All rights reserved.
Copyright (c) 2022 Cisco Systems, Inc. All rights reserved.
Copyright (c) 2022 IBM Corporation. All rights reserved.
Copyright (c) 2023 Jeffrey M. Squyres. All rights reserved.
Copyright (c) 2024 Triad National Security, LLC. All rights reserved.
$COPYRIGHT$
Additional copyrights may follow
$HEADER$
General command line options
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The following general command line options are available.
* ``-h`` | ``--help``: This help message
* ``-v`` | ``--verbose``: Enable extra verbose output
* ``-V`` | ``--version``: Print version and exit
Launch options
^^^^^^^^^^^^^^
* ``--default-hostfile <filename>``: Provide a default hostfile.
* ``-H`` | ``--host <list>``: Comma-delimited list of hosts on which to
to invoke processes.
:ref:`See below for details <label-schizo-ompi-host>`.
* ``--hostfile <filename>``: Provide a hostfile.
:ref:`See below for details <label-schizo-ompi-hostfile>`.
* ``--machinefile <filename>``: Synonym for ``--hostfile``.
* ``--mca <key> <value>``: Pass context-specific MCA parameters; they
are considered global if ``--gmca`` is not used and only one context
is specified (``<key>`` is the parameter name; ``<value>`` is the
parameter value).
* ``--path <path>``: Path to be used to look for executables to start
processes.
* ``--pmixmca <key> <value>``: Pass context-specific PMIx MCA
parameters; they are considered global if only one context is
specified (``<key>>`` is the parameter name; ``<value>`` is the
parameter value).
:ref:`See below for details <label-schizo-ompi-pmixmca>`.
* ``--preload-files <list>``: Preload the comma-separated list of
files to the remote machines current working directory before
starting the remote process.
* ``--prtemca <key> <value>``: Pass context-specific PRTE MCA
parameters to the DVM.
:ref:`See below for details <label-schizo-ompi-prtemca>`.
* ``--pset <name>``: User-specified name assigned to the processes in
their given application.
* ``--rankfile <filename>``: Name of file to specify explicit task
mapping.
* ``--runtime-options <list>``: List of options to pass to the launcher.
:ref:`See below for details <label-schizo-ompi-runtime-options>`.
* ``-s`` | ``--preload-binary``: Preload the binary on the remote
machine before starting the remote process.
* ``--set-cwd-to-session-dir``: Set the working directory of the
started processes to their session directory.
* ``--show-progress``: Output a brief periodic report on launch
progress.
* ``--wd <dir>``: Synonym for ``--wdir``.
* ``--wdir <dir>``: Set the working directory of the started processes.
* ``-x <var>``: Export a environment variable, optionally specifying a value.
:ref:`See below for details <label-schizo-ompi-x>`.
Mapping, ranking, and binding options
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* ``--map-by <value>``: Controls how processes are distributed within
a host. :ref:`See below for details <label-schizo-ompi-map-by>`.
* ``--rank-by <value>``: Controls how processes are ordered in
``MPI_COMM_WORLD``. :ref:`See below for details
<label-schizo-ompi-rank-by>`.
* ``--bind-to <value>``: Controls the resources to which processes are
bound within a host. :ref:`See below for details
<label-schizo-ompi-bind-to>`.
Output options
^^^^^^^^^^^^^^
* ``--output <list>``: Comma-delimited list of options that control
how output is generated. :ref:`See below for details
<label-schizo-ompi-output>`.
* ``--report-child-jobs-separately``: Return the exit status of the
primary job only.
* ``--stream-buffering <value>``: Control how output is buffered.
:ref:`See below for details <label-schizo-ompi-stream-buffering>`.
* ``--xterm <list>``: Create a new xterm window and display output
from the specified ranks there.
Input options
^^^^^^^^^^^^^
* ``--stdin <list>``: Specify procs to receive stdin; alloable values
in ``<list>`` are: comma-delimited list of integer ranks, ``all``,
``none``. Defaults to ``0``, indicating rank 0.
Specific options
^^^^^^^^^^^^^^^^
* ``--allow-run-as-root``: Allow execution as root **(STRONGLY
DISCOURAGED)**. :ref:`See below for details
<label-schizo-ompi-allow-run-as-root>`.
* ``--daemonize``: Daemonize the DVM daemons into the background.
* ``--forward-signals <list>``: Comma-delimited list of additional
signals (names or integers) to forward to application
processes.
:ref:`See below for details <label-schizo-ompi-forward-signals>`.
* ``--launch-agent <agent>``: Name of daemon executable used to start
processes on remote nodes (default: ``prted``).
* ``--max-vm-size <size>``: Number of daemons to start.
* ``--noprefix``: Disable automatic ``--prefix`` behavior.
:ref:`See below for details <label-schizo-ompi-noprefix>`.
* ``--prefix <prefix>``: Prefix to be used to look for RTE executables.
* ``--set-sid``: Direct the DVM daemons to separate from the current
session.
* ``--singleton <id>``: ID of the singleton process that started us.
* ``--tmpdir <dir>``: Set the root for the session directory tree.
* ``--tune <filename>``: File(s) containing MCA params for tuning DVM
operations.
:ref:`See below for details <label-schizo-ompi-tune>`.
MPI Options
^^^^^^^^^^^
* ``--initial-errhandler``: Specify the initial error handler that is
attached to predefined communicators during the first MPI call.
* ``--memory-alloc-kinds``: Value is a comma separated list of
memory allocation kinds.
* ``--display-comm``: Display table of communication methods between
MPI_COMM_WORLD ranks during MPI_Init
* ``--display-comm-finalize``: Display table of communication methods
between ranks during MPI_Finalize
* ``--soft``: This option does nothing, but is mandated by the MPI
standard
* ``--arch <filename>``: This option does nothing, but is mandated by
the MPI standard
* ``--file <filename>``: This option does nothing, but is mandated by
the MPI standard
Debugger / tool options
^^^^^^^^^^^^^^^^^^^^^^^
* ``--keepalive <filename>``: Named pipe filename to monitor |mdash|
DVM will terminate upon closure
* ``--report-pid <value>``: Print out PID on stdout (``-``), stderr
(``+``), or a filename (anything else)
* ``--report-uri <value>``: Print out URI on stdout (``-``), stderr
(``+``), or a filename (anything else)
* ``--stop-on-exec <value>``: If supported, stop each specified
process at start of execution
* ``--stop-in-init <value>``: Direct the specified processes to stop
in the MPI initialization function
* ``--stop-in-app <value>``: Direct the specified processes to stop at
an application-controlled location
Debug options
^^^^^^^^^^^^^
* ``--debug-daemons``: Debug daemons |mdash| if not set, the "verbose"
setting will be limited to the DVM controller to reduce clutter.
:ref:`See below for details <label-schizo-ompi-debug-daemons>`.
* ``--debug-daemons-file``: Enable debugging of any PRTE daemons used
by this application, storing their verbose output in files.
:ref:`See below for details <label-schizo-ompi-debug-daemons-file>`.
* ``--display <value>``: Comma-delimited list of options for
displaying information about the allocation and job. Allowed values
for ``<value>``: ``allocation``, ``bind``, ``map``, ``map-devel``,
``topo``.
:ref:`See below for details <label-schizo-ompi-display>`.
* ``--get-stack-traces``: If a timeout occurs, get stack traces of all
application processes.
* ``--leave-session-attached``: Do not discard stdout/stderr of remote
PRTE daemons.
:ref:`See below for details <label-schizo-ompi-leave-session-attached>`.
* ``--report-state-on-timeout``: Report all job and process states
upon timeout.
* ``--spawn-timeout <seconds>``: Timeout the job if spawn takes more
than the specified number of seconds.
* ``--test-suicide <seconds>``: Suicide instead of clean abort after
delay.
* ``--timeout <seconds>``: Timeout the job after the specified number
of seconds.
* ``--output-proctable <value>``: Print the complete proctable to
stdout after launch; allowed ``<value>`` values are ``+``, ``-``, or
a filename.
Fault Tolerance Options
^^^^^^^^^^^^^^^^^^^^^^^
These options are only available is Open MPI / PRRTE were compiled
with Fault Tolerance functionality enabled.
* ``--enable-recovery``: Enable recovery from process failure (Default
= disabled)
* ``--max-restarts``: Max number of times to restart a failed process
* ``--disable-recovery``: Disable recovery (resets all recovery
options to off)
* ``--continuous``: Job is to run until explicitly terminated
* ``--with-ft``: Specify the type(s) of error handling that the
application will use.
Details of individual command line options
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The sections below offer more detail than the abbreviated lists,
above.
.. _label-schizo-ompi-allow-run-as-root:
The ``--allow-run-as-root`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-allow-run-as-root.rst
.. _label-schizo-ompi-bind-to:
The ``--bind-to`` option
~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-bind-to.rst
.. _label-schizo-ompi-debug-daemons:
The ``--debug-daemons`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-debug-daemons.rst
.. _label-schizo-ompi-debug-daemons-file:
The ``--debug-daemons-file`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-debug-daemons-file.rst
.. _label-schizo-ompi-display:
The ``--display`` option
~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-display.rst
.. _label-schizo-ompi-forward-signals:
The ``--forward-signals`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-forward-signals.rst
.. _label-schizo-ompi-host:
The ``--host`` option
~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-dash-host.rst
.. _label-schizo-ompi-hostfile:
The ``--hostfile`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-dvm-hostfile.rst
.. _label-schizo-ompi-leave-session-attached:
The ``--leave-session-attached`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-leave-session-attached.rst
.. _label-schizo-ompi-map-by:
The ``--map-by`` option
~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-map-by.rst
.. _label-schizo-ompi-output:
The ``--output`` option
~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-output.rst
.. _label-schizo-ompi-pmixmca:
The ``--pmixmca`` option
~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-pmixmca.rst
.. _label-schizo-ompi-prefix:
The ``--prefix`` option
~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-prefix.rst
.. _label-schizo-ompi-prtemca:
The ``--prtemca`` option
~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-prtemca.rst
.. _label-schizo-ompi-noprefix:
The ``--noprefix`` option
~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-noprefix.rst
.. _label-schizo-ompi-rank-by:
The ``--rank-by`` option
~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-rank-by.rst
.. _label-schizo-ompi-runtime-options:
The ``--runtime-options`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-runtime-options.rst
.. _label-schizo-ompi-stream-buffering:
The ``--stream-buffering`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-stream-buffering.rst
.. _label-schizo-ompi-tune:
The ``--tune`` option
~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-tune.rst
.. _label-schizo-ompi-x:
The ``-x`` option
~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/cli-x.rst
Deprecated command line options
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The following command line options are deprecated, and should
generally not be used. They may be removed in a future release of
Open MPI.
The ``--bind-to-core`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-bind-to-core.rst
The ``--display-allocation`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-display-allocation.rst
The ``--display-devel-allocation`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-display-devel-allocation.rst
The ``--display-devel-map`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-display-devel-map.rst
The ``--display-map`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-display-map.rst
The ``--display-topo`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-display-topo.rst
The ``--gmca`` option
~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-gmca.rst
The ``--mca`` option
~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-mca.rst
The ``--merge-stderr-to-stdout`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-merge-stderr-to-stdout.rst
The ``--output-directory`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-output-directory.rst
The ``--output-filename`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-output-filename.rst
The ``--report-bindings`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-report-bindings.rst
The ``--tag-output`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-tag-output.rst
The ``--timestamp-output`` option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-timestamp-output.rst
The ``--xml`` option
~~~~~~~~~~~~~~~~~~~~
.. include:: /prrte-rst-content/deprecated-xml.rst