mac mpi

macx64/mpi/mpich/share/man/man1/hydra_nameserver.1

@@ -0,0 +1,10 @@
+.TH hydra_nameserver 1 "6/5/2019" " " "HYDRA"
+.SH NAME
+hydra_nameserver \-  Internal executable used by Hydra 
+.SH DESCRIPTION
+This executable is designed to not be run directly by users, but to
+support the Hydra process manager. As such, no documentation will be
+provided here.
+
+.SH LOCATION
+/tmp/kS7n0Mi0xa/mpich-3.3.1/src/pm/hydra/tools/nameserver/hydra_nameserver.txt

macx64/mpi/mpich/share/man/man1/hydra_persist.1

@@ -0,0 +1,10 @@
+.TH hydra_persist 1 "6/5/2019" " " "HYDRA"
+.SH NAME
+hydra_persist \-  Internal executable used by Hydra 
+.SH DESCRIPTION
+This executable is designed to not be run directly by users, but to
+support the Hydra process manager. As such, no documentation will be
+provided here.
+
+.SH LOCATION
+/tmp/kS7n0Mi0xa/mpich-3.3.1/src/pm/hydra/tools/bootstrap/persist/hydra_persist.txt

macx64/mpi/mpich/share/man/man1/hydra_pmi_proxy.1

@@ -0,0 +1,10 @@
+.TH hydra_pmi_proxy 1 "6/5/2019" " " "HYDRA"
+.SH NAME
+hydra_pmi_proxy \-  Internal exectuable used by Hydra 
+.SH DESCRIPTION
+This executable is designed to not be run directly by users, but to
+support the Hydra process manager. As such, no documentation will be
+provided here.
+
+.SH LOCATION
+/tmp/kS7n0Mi0xa/mpich-3.3.1/src/pm/hydra/pm/pmiserv/hydra_pmi_proxy.txt

macx64/mpi/mpich/share/man/man1/mpicc.1

@@ -0,0 +1,217 @@
+.TH mpicc 1 "6/5/2019" " " "MPI"
+.SH NAME
+mpicc \-  Compiles and links MPI programs written in C 
+.SH DESCRIPTION
+This command can be used to compile and link MPI programs written in
+C.  It provides the options and any special libraries that are
+needed to compile and link MPI programs.
+
+It is important to use this command, particularly when linking programs,
+as it provides the necessary libraries.
+
+.SH COMMAND LINE ARGUMENTS
+.PD 0
+.TP
+.B -show      
+- Show the commands that would be used without
+running them
+.PD 1
+.PD 0
+.TP
+.B -help      
+- Give short help
+.PD 1
+.PD 0
+.TP
+.B -cc=name   
+- Use compiler 
+.I name
+instead of the default choice.  Use
+this only if the compiler is compatible with the MPICH
+library (see below)
+.PD 1
+.PD 0
+.TP
+.B -config=name 
+- Load a configuration file for a particular compiler.
+This allows a single 
+.I mpicc
+command to be used with 
+multiple compilers.
+.PD 1
+.PD 0
+.TP
+.B -compile_info 
+- Show the steps for compiling a program.  This option
+can be used to see what options and include paths are
+used by mpicc.
+.PD 1
+.PD 0
+.TP
+.B -link_info 
+- Show the steps for linking a program.  This option
+can be used to see what options and libraries are used by
+mpicc.
+.PD 1
+.PD 0
+.TP
+.B -profile=name 
+- Use the MPI profiling given by name.  See below for
+details
+.PD 1
+.PD 0
+.TP
+.B -echo      
+- Show exactly what this program is doing.
+This option should normally not be used.
+.PD 1
+.PD 0
+.TP
+.B -static
+- mpi - Use a statically compile MPI library, but shared libraries
+for all of the other dependencies.
+.PD 1
+.PD 0
+.TP
+.B others     
+- are passed to the compiler or linker.  For example, 
+.I \\-c
+causes files to be compiled, 
+.I \\-g
+selects compilation with
+debugging on most systems, and 
+.I \\-o name
+causes linking
+with the output executable given the name 
+.I name
+\&.
+
+.PD 1
+
+.SH ENVIRONMENT VARIABLES
+The environment variable 
+.I MPICH_CC
+may be used
+to select different C compiler and linker.  Note that since
+MPICH is built with a particular C and Fortran compiler, changing the
+compilers used can cause problems.  Use this only if you could intermix
+code compiled with the different compilers.
+
+The environment variable 
+.I MPICC_PROFILE
+specifies a profile library
+and has the same effect as if 
+.I \\-profile=$MPICC_PROFILE
+were used as
+an argument to 
+.I mpicc
+\&.
+See the discussion of 
+.I \\-profile
+below for more
+details.
+
+.SH COMPATIBLE COMPILERS
+The MPI library may be used with any compiler that uses the same
+lengths for basic data objects (such as 
+.I long double
+) and that
+uses compatible run-time libraries.  On many systems, the various
+compilers are compatible and may be used interchangably.  There are
+exceptions; if you use the 
+.I MPICH_CC
+environment variable or the
+.I \\-cc=name
+command-line argument to override the choice of compiler
+and encounter problems, try reconfiguring MPICH with the new compiler
+and installing MPICH in a separate location.  See the installation manual
+for more details.
+
+.SH EXAMPLES
+To compile a single file 
+.I foo.c
+, use
+.nf
+mpicc -c foo.c
+.fi
+
+
+To link the output and make an executable, use
+.nf
+mpicc -o foo foo.o
+.fi
+
+Combining compilation and linking in a single command
+.nf
+mpicc -o foo foo.c
+.fi
+
+is a convenient way to build simple programs.
+
+.SH SELECTING A PROFILING LIBRARY
+The 
+.I \\-profile=name
+argument allows you to specify an MPI profiling
+library to be used.  
+.I name
+can have two forms:
+
+.br
+A library in the same directory as the MPI library
+.br
+The name of a profile configuration file
+.br
+
+If 
+.I name
+is a library, then this library is included before the MPI
+library.  This allows the simple use of libraries that make use of the
+MPI profiling interface and that are installed in the same directory as
+the MPI library.
+
+If 
+.I name.conf
+is the name of a file in the sysconfdir directory, then this
+is read and may define the following variables:
+.PD 0
+.TP
+.B PROFILE_PRELIB 
+- Libraries (and paths) to include before the MPI library
+.PD 1
+.PD 0
+.TP
+.B PROFILE_POSTLIB 
+- Libraries to include after the MPI library
+.PD 1
+.PD 0
+.TP
+.B PROFILE_INCPATHS 
+- C preprocessor arguments for any include files
+For example, to add 
+.I /usr/local/myprof/include
+to the include path and
+the library 
+.I libmyprof.a
+in 
+.I /usr/local/myprof/lib
+to the link step, 
+you could create the file 
+.I myprof.conf
+with the lines
+.PD 1
+
+.nf
+PROFILE_PRELIB="-L/usr/local/myprof/lib -lmyprof"
+PROFILE_INCPATHS="-I/usr/local/myprof/include"
+.fi
+
+and place it in the sysconfdir directory (this directory is set at
+configure time when MPICH is built).  Then using the command-line
+argument 
+.I \\-profile=myprof
+will cause these
+definitions to be added to the relevant compile commands.
+
+.SH SEE ALSO
+mpicxx, mpifort, mpiexec
+.br

macx64/mpi/mpich/share/man/man1/mpicxx.1

@@ -0,0 +1,217 @@
+.TH mpicxx 1 "6/5/2019" " " "MPI"
+.SH NAME
+mpicxx \-  Compiles and links MPI programs written in C++ 
+.SH DESCRIPTION
+This command can be used to compile and link MPI programs written in
+C++.  It provides the options and any special libraries that are
+needed to compile and link MPI programs.
+
+It is important to use this command, particularly when linking programs,
+as it provides the necessary libraries.
+
+.SH COMMAND LINE ARGUMENTS
+.PD 0
+.TP
+.B -show      
+- Show the commands that would be used without
+running them
+.PD 1
+.PD 0
+.TP
+.B -help      
+- Give short help
+.PD 1
+.PD 0
+.TP
+.B -cxx=name   
+- Use compiler 
+.I name
+instead of the default choice.  Use
+this only if the compiler is compatible with the MPICH
+library (see below)
+.PD 1
+.PD 0
+.TP
+.B -config=name 
+- Load a configuration file for a particular compiler.
+This allows a single 
+.I mpicxx
+command to be used with 
+multiple compilers.
+.PD 1
+.PD 0
+.TP
+.B -compile_info 
+- Show the steps for compiling a program.  This option
+can be used to see what options and include paths are
+used by mpicxx.
+.PD 1
+.PD 0
+.TP
+.B -link_info 
+- Show the steps for linking a program.  This option
+can be used to see what options and libraries are used by
+mpicxx.
+.PD 1
+.PD 0
+.TP
+.B -profile=name 
+- Use the MPI profiling given by name.  See below for
+details
+.PD 1
+.PD 0
+.TP
+.B -echo      
+- Show exactly what this program is doing.
+This option should normally not be used.
+.PD 1
+.PD 0
+.TP
+.B -static
+- mpi - Use a statically compile MPI library, but shared libraries
+for all of the other dependencies.
+.PD 1
+.PD 0
+.TP
+.B others     
+- are passed to the compiler or linker.  For example, 
+.I \\-c
+causes files to be compiled, 
+.I \\-g
+selects compilation with
+debugging on most systems, and 
+.I \\-o name
+causes linking
+with the output executable given the name 
+.I name
+\&.
+
+.PD 1
+
+.SH ENVIRONMENT VARIABLES
+The environment variables 
+.I MPICH_CXX
+may be used
+to select different C++ compiler and linker.  Note that since
+MPICH is built with a particular C and Fortran compiler, changing the
+compilers used can cause problems.  Use this only if you could intermix
+code compiled with the different compilers.
+
+The environment variable 
+.I MPICC_PROFILE
+specifies a profile library
+and has the same effect as if 
+.I \\-profile=$MPICC_PROFILE
+were used as
+an argument to 
+.I mpicc
+\&.
+See the discussion of 
+.I \\-profile
+below for more
+details.
+
+.SH COMPATIBLE COMPILERS
+The MPI library may be used with any compiler that uses the same
+lengths for basic data objects (such as 
+.I long double
+) and that
+uses compatible run-time libraries.  On many systems, the various
+compilers are compatible and may be used interchangably.  There are
+exceptions; if you use the 
+.I MPICH_CXX
+environment variable or the
+.I \\-cxx=name
+command-line argument to override the choice of compiler
+and encounter problems, try reconfiguring MPICH with the new compiler,
+and installing MPICH in a separate location.  See the installation manual
+for more details.
+
+.SH EXAMPLES
+To compile a single file 
+.I foo.c
+, use
+.nf
+mpicxx -c foo.cxx
+.fi
+
+
+To link the output and make an executable, use
+.nf
+mpicxx -o foo foo.o
+.fi
+
+Combining compilation and linking in a single command
+.nf
+mpicxx -o foo foo.cxx
+.fi
+
+is a convenient way to build simple programs.
+
+.SH SELECTING A PROFILING LIBRARY
+The 
+.I \\-profile=name
+argument allows you to specify an MPI profiling
+library to be used.  
+.I name
+can have two forms:
+
+.br
+A library in the same directory as the MPI library
+.br
+The name of a profile configuration file
+.br
+
+If 
+.I name
+is a library, then this library is included before the MPI
+library.  This allows the simple use of libraries that make use of the
+MPI profiling interface and that are installed in the same directory as
+the MPI library.
+
+If 
+.I name.conf
+is the name of a file in the sysconfdir directory, then this
+is read and may define the following variables:
+.PD 0
+.TP
+.B PROFILE_PRELIB 
+- Libraries (and paths) to include before the MPI library
+.PD 1
+.PD 0
+.TP
+.B PROFILE_POSTLIB 
+- Libraries to include after the MPI library
+.PD 1
+.PD 0
+.TP
+.B PROFILE_INCPATHS 
+- C preprocessor arguments for any include files
+For example, to add 
+.I /usr/local/myprof/include
+to the include path and
+the library 
+.I libmyprof.a
+in 
+.I /usr/local/myprof/lib
+to the link step, 
+you could create the file 
+.I myprof.conf
+with the lines
+.PD 1
+
+.nf
+PROFILE_PRELIB="-L/usr/local/myprof/lib -lmyprof"
+PROFILE_INCPATHS="-I/usr/local/myprof/include"
+.fi
+
+and place it in the sysconfdir directory (this directory is set at
+configure time when MPICH is built).  Then using the command-line
+argument 
+.I \\-profile=myprof
+will cause these
+definitions to be added to the relevant compile commands.
+
+.SH SEE ALSO
+mpicc, mpifort, mpiexec
+.br

macx64/mpi/mpich/share/man/man1/mpiexec.1

@@ -0,0 +1,321 @@
+.TH mpiexec 1 "6/5/2019" " " "MPI"
+.SH NAME
+mpiexec \-  Run an MPI program 
+.SH SYNOPSIS
+.nf
+.fi
+.nf
+mpiexec args executable pgmargs [ : args executable pgmargs ... ]
+.fi
+
+where 
+.I args
+are command line arguments for 
+.I mpiexec
+(see below),
+.I executable
+is the name of an executable MPI program, and 
+.I pgmargs
+are command line arguments for the executable.  Multiple executables
+can be specified by using the colon notation (for MPMD - Multiple Program
+Multiple Data applications).   For example, the following command will run
+the MPI program 
+.I a.out
+on 4 processes:
+.nf
+mpiexec -n 4 a.out
+.fi
+
+
+The MPI standard specifies the following arguments and their meanings:
+
+.PD 0
+.TP
+.B -n <np> 
+- Specify the number of processes to use
+.PD 1
+.PD 0
+.TP
+.B -host <hostname> 
+- Name of host on which to run processes
+.PD 1
+.PD 0
+.TP
+.B -arch <architecture name> 
+- Pick hosts with this architecture type
+.PD 1
+.PD 0
+.TP
+.B -wdir <working directory> 
+- cd to this one 
+.B before
+running executable
+.PD 1
+.PD 0
+.TP
+.B -path <pathlist> 
+- use this to find the executable
+.PD 1
+.PD 0
+.TP
+.B -soft <triplets> 
+- comma separated triplets that specify requested numbers
+of processes (see the MPI-2 specification for more details)
+.PD 1
+.PD 0
+.TP
+.B -file <name> 
+- implementation-defined specification file
+.PD 1
+.PD 0
+.TP
+.B -configfile <name> 
+- file containing specifications of host/program,
+one per line, with # as a comment indicator, e.g., the usual
+mpiexec input, but with ":" replaced with a newline.  That is,
+the configfile contains lines with -soft, -n etc.
+.PD 1
+
+Additional arguments that are specific to the MPICH implementation
+are discussed below.
+
+Note that not all of these parameters are meaningful for all
+systems.  For example, the 
+.I gforker
+version of 
+.I mpiexec
+creates all
+processes on the same system on which it is running; in that case, the
+.I \\-arch
+and 
+.I \\-host
+options are ignored.
+
+The colon character (
+.I :
+) may be used to separate different executables
+for MPMD (multiple program multiple data) programming.  For example,
+to run the program 
+.I ocean
+on 4 processes and 
+.I air
+on 8 processes, use:
+
+.nf
+mpiexec -n 4 ocean : -n 8 air
+.fi
+
+
+
+.SH MPICH-SPECIFIC ARGUMENTS
+
+Many of the implementations of process managers in MPICH support the
+following arguments to 
+.I mpiexec
+:
+
+.PD 0
+.TP
+.B -np <num> 
+- A synonym for the standard 
+.I \\-n
+argument
+.PD 1
+.PD 0
+.TP
+.B -env <name> <value> 
+- Set the environment variable 
+.I <name>
+to 
+.I <value>
+for
+the processes being run by 
+.I mpiexec
+
+.PD 1
+.PD 0
+.TP
+.B -envnone 
+- Pass no environment variables (other than ones specified with
+other 
+.I \\-env
+or 
+.I \\-genv
+arguments) to the processes being run by 
+.I mpiexec
+\&.
+
+By default, all environment
+variables are provided to each MPI process (rationale: principle of
+least surprise for the user)
+.PD 1
+.PD 0
+.TP
+.B -envlist <list> 
+- Pass the listed environment variables (names separated
+by commas), with their current values, to the processes being run by
+.I mpiexec
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B -genv <name> <value> 
+- The 
+.I \\-genv
+options have the same meaning as their
+corresponding 
+.I \\-env
+version, except they apply to all executables, not just
+the current executable (in the case that the colon syntax is used to specify
+multiple execuables).
+.PD 1
+.PD 0
+.TP
+.B -genvnone 
+- Like 
+.I \\-envnone
+, but for all executables
+.PD 1
+.PD 0
+.TP
+.B -genvlist <list> 
+- Like 
+.I \\-envlist
+, but for all executables
+.PD 1
+.PD 0
+.TP
+.B -usize <n> 
+- Specify the value returned for the value of the attribute
+.I MPI_UNIVERSE_SIZE
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B -l 
+- Label standard out and standard error (
+.I stdout
+and 
+.I stderr
+) with
+the rank of the process
+.PD 1
+.PD 0
+.TP
+.B -maxtime <n> 
+- Set a timelimit of 
+.I <n>
+seconds.
+.PD 1
+.PD 0
+.TP
+.B -exitinfo 
+- Provide more information on the reason each process exited if
+there is an abnormal exit
+.PD 1
+
+.SH ENVIRONMENT VARIABLES FOR MPIEXEC
+The following environment variables are understood by some versions of
+.I mpiexec
+\&.
+The command line arguments have priority over these; that is,
+if both the environment variable and command line argument are used, the
+value specified by the command line argument is used.
+
+.PD 0
+.TP
+.B MPIEXEC_TIMEOUT  
+- Maximum running time in seconds.  
+.I mpiexec
+will
+terminate MPI programs that take longer than the value specified by
+.I MPIEXEC_TIMEOUT
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B MPIEXEC_UNIVERSE_SIZE 
+- Set the universe size
+.PD 1
+.PD 0
+.TP
+.B MPIEXEC_PORT_RANGE 
+- Set the range of ports that 
+.I mpiexec
+will use 
+in communicating with the processes that it starts.  The format of 
+this is 
+.I <low>:<high>
+\&.
+For example, to specify any port between
+10000 and 10100, use 
+.I 10000:10100
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B MPICH_PORT_RANGE 
+- Has the same meaning as 
+.I MPIEXEC_PORT_RANGE
+and 
+is used if 
+.I MPIEXEC_PORT_RANGE
+is not set.
+.PD 1
+.PD 0
+.TP
+.B MPIEXEC_PREFIX_DEFAULT 
+- If this environment variable is set, output
+to standard output is prefixed by the rank in 
+.I MPI_COMM_WORLD
+of the 
+process and output to standard error is prefixed by the rank and the 
+text 
+.I (err)
+; both are followed by an angle bracket (
+.I >
+).  If
+this variable is not set, there is no prefix.
+.PD 1
+.PD 0
+.TP
+.B MPIEXEC_PREFIX_STDOUT 
+- Set the prefix used for lines sent to standard
+output.  A 
+.I %d
+is replaced with the rank in 
+.I MPI_COMM_WORLD
+; a 
+.I %w
+is 
+replaced with an indication of which 
+.I MPI_COMM_WORLD
+in MPI jobs that
+involve multiple 
+.I MPI_COMM_WORLD
+s (e.g., ones that use 
+.I MPI_Comm_spawn
+or
+.I MPI_Comm_connect
+).
+.PD 1
+.PD 0
+.TP
+.B MPIEXEC_PREFIX_STDERR 
+- Like 
+.I MPIEXEC_PREFIX_STDOUT
+, but for standard error.
+.PD 1
+
+.SH RETURN STATUS
+.I mpiexec
+returns the maximum of the exit status values of all of the
+processes created by 
+.I mpiexec
+\&.
+
+

macx64/mpi/mpich/share/man/man1/mpif77.1

@@ -0,0 +1,203 @@
+.TH mpif77 1 "6/5/2019" " " "MPI"
+.SH NAME
+mpif77 \-  Compiles and links MPI programs written in Fortran 77 
+.SH DESCRIPTION
+This command can be used to compile and link MPI programs written in
+Fortran.  It provides the options and any special libraries that are
+needed to compile and link MPI programs.
+
+It is important to use this command, particularly when linking programs,
+as it provides the necessary libraries.
+
+.SH COMMAND LINE ARGUMENTS
+.PD 0
+.TP
+.B -show      
+- Show the commands that would be used without
+running them
+.PD 1
+.PD 0
+.TP
+.B -help      
+- Give short help
+.PD 1
+.PD 0
+.TP
+.B -f77=name   
+- Use compiler 
+.I name
+instead of the default choice.  Use
+this only if the compiler is compatible with the MPICH
+library (see below)
+.PD 1
+.PD 0
+.TP
+.B -config=name 
+- Load a configuration file for a particular compiler.
+This allows a single 
+.I mpif77
+command to be used with 
+multiple compilers.
+.PD 1
+.PD 0
+.TP
+.B -compile_info 
+- Show the steps for compiling a program.  This option
+can be used to see what options and include paths are
+used by mpif77.
+.PD 1
+.PD 0
+.TP
+.B -link_info 
+- Show the steps for linking a program.  This option
+can be used to see what options and libraries are used by
+mpif77.
+.PD 1
+.PD 0
+.TP
+.B -profile=name 
+- Use the MPI profiling given by name.  See below for
+details
+.PD 1
+.PD 0
+.TP
+.B -echo      
+- Show exactly what this program is doing.
+This option should normally not be used.
+.PD 1
+.PD 0
+.TP
+.B -static
+- mpi - Use a statically compile MPI library, but shared libraries
+for all of the other dependencies.
+.PD 1
+.PD 0
+.TP
+.B others     
+- are passed to the compiler or linker.  For example, 
+.I \\-c
+causes files to be compiled, 
+.I \\-g
+selects compilation with
+debugging on most systems, and 
+.I \\-o name
+causes linking
+with the output executable given the name 
+.I name
+\&.
+
+.PD 1
+
+.SH ENVIRONMENT VARIABLES
+The environment variables 
+.I MPICH_F77
+may be used
+to select different Fortran compiler and linker.  Note that since
+MPICH is built with a particular C and Fortran compiler, change the
+compilers used can cause problems.  Use this only if you could intermix
+code compiled with the different compilers.
+
+.SH COMPATIBLE COMPILERS
+The MPI library may be used with any compiler that uses the same
+lengths for basic data objects (such as 
+.I long double
+) and that
+uses compatible run-time libraries.  On many systems, the various
+compilers are compatible and may be used interchangably.  There are
+exceptions; if you use the 
+.I MPICH_F77
+environment variable or the
+.I \\-f77=name
+command-line argument to override the choice of compiler
+and encounter problems, try reconfiguring MPICH with the new compiler
+and installing MPICH in a separate location.  See the installation manual
+for more details.
+
+.SH EXAMPLES
+To compile a single file 
+.I foo.f
+, use
+.nf
+mpif77 -c foo.f
+.fi
+
+
+To link the output and make an executable, use
+.nf
+mpif77 -o foo foo.o
+.fi
+
+Combining compilation and linking in a single command
+.nf
+mpif77 -o foo foo.f
+.fi
+
+is a convenient way to build simple programs.
+
+.SH SELECTING A PROFILING LIBRARY
+The 
+.I \\-profile=name
+argument allows you to specify an MPI profiling
+library to be used.  
+.I name
+can have two forms:
+
+.br
+A library in the same directory as the MPI library
+.br
+The name of a profile configuration file
+.br
+
+If 
+.I name
+is a library, then this library is included before the MPI
+library.  This allows the simple use of libraries that make use of the
+MPI profiling interface and that are installed in the same directory as
+the MPI library.
+
+If 
+.I name.conf
+is the name of a file in the sysconfdir directory, then this
+is read and may define the following variables:
+.PD 0
+.TP
+.B PROFILE_PRELIB 
+- Libraries (and paths) to include before the MPI library
+.PD 1
+.PD 0
+.TP
+.B PROFILE_POSTLIB 
+- Libraries to include after the MPI library
+.PD 1
+.PD 0
+.TP
+.B PROFILE_INCPATHS 
+- C preprocessor arguments for any include files
+For example, to add 
+.I /usr/local/myprof/include
+to the include path and
+the library 
+.I libmyprof.a
+in 
+.I /usr/local/myprof/lib
+to the link step, 
+you could create the file 
+.I myprof.conf
+with the lines
+.PD 1
+
+.nf
+PROFILE_PRELIB="-L/usr/local/myprof/lib -lmyprof"
+PROFILE_INCPATHS="-I/usr/local/myprof/include"
+.fi
+
+and place it in the sysconfdir directory (this directory is set at
+configure time when MPICH is built).  Then using the command-line
+argument 
+.I \\-profile=myprof
+will cause these
+definitions to be added to the relevant compile commands.
+
+.SH SEE ALSO
+mpicc, mpicxx, mpifort, mpiexec
+.br

macx64/mpi/mpich/share/man/man1/mpifort.1

@@ -0,0 +1,203 @@
+.TH mpifort 1 "6/5/2019" " " "MPI"
+.SH NAME
+mpifort \-  Compiles and links MPI programs written in Fortran 90 
+.SH DESCRIPTION
+This command can be used to compile and link MPI programs written in
+Fortran.  It provides the options and any special libraries that are
+needed to compile and link MPI programs.
+
+It is important to use this command, particularly when linking programs,
+as it provides the necessary libraries.
+
+.SH COMMAND LINE ARGUMENTS
+.PD 0
+.TP
+.B -show      
+- Show the commands that would be used without
+running them
+.PD 1
+.PD 0
+.TP
+.B -help      
+- Give short help
+.PD 1
+.PD 0
+.TP
+.B -fc=name   
+- Use compiler 
+.I name
+instead of the default choice.  Use
+this only if the compiler is compatible with the MPICH
+library (see below)
+.PD 1
+.PD 0
+.TP
+.B -config=name 
+- Load a configuration file for a particular compiler.
+This allows a single 
+.I mpifort
+command to be used with
+multiple compilers.
+.PD 1
+.PD 0
+.TP
+.B -compile_info 
+- Show the steps for compiling a program.  This option
+can be used to see what options and include paths are
+used by mpifort.
+.PD 1
+.PD 0
+.TP
+.B -link_info 
+- Show the steps for linking a program.  This option
+can be used to see what options and libraries are used by
+mpifort.
+.PD 1
+.PD 0
+.TP
+.B -profile=name 
+- Use the MPI profiling given by name.  See below for
+details
+.PD 1
+.PD 0
+.TP
+.B -echo      
+- Show exactly what this program is doing.
+This option should normally not be used.
+.PD 1
+.PD 0
+.TP
+.B -static
+- mpi - Use a statically compile MPI library, but shared libraries
+for all of the other dependencies.
+.PD 1
+.PD 0
+.TP
+.B others     
+- are passed to the compiler or linker.  For example, 
+.I \\-c
+causes files to be compiled, 
+.I \\-g
+selects compilation with
+debugging on most systems, and 
+.I \\-o name
+causes linking
+with the output executable given the name 
+.I name
+\&.
+
+.PD 1
+
+.SH ENVIRONMENT VARIABLES
+The environment variables 
+.I MPICH_FC
+may be used
+to select different Fortran compiler and linker.  Note that since
+MPICH is built with a particular C and Fortran compiler, change the
+compilers used can cause problems.  Use this only if you could intermix
+code compiled with the different compilers.
+
+.SH COMPATIBLE COMPILERS
+The MPI library may be used with any compiler that uses the same
+lengths for basic data objects (such as 
+.I long double
+) and that
+uses compatible run-time libraries.  On many systems, the various
+compilers are compatible and may be used interchangably.  There are
+exceptions; if you use the 
+.I MPICH_FC
+environment variable or the
+.I \\-fc=name
+command-line argument to override the choice of compiler
+and encounter problems, try reconfiguring MPICH with the new compiler
+and installing MPICH in a separate location.  See the installation manual
+for more details.
+
+.SH EXAMPLES
+To compile a single file 
+.I foo.f
+, use
+.nf
+mpifort -c foo.f
+.fi
+
+
+To link the output and make an executable, use
+.nf
+mpifort -o foo foo.o
+.fi
+
+Combining compilation and linking in a single command
+.nf
+mpifort -o foo foo.f
+.fi
+
+is a convenient way to build simple programs.
+
+.SH SELECTING A PROFILING LIBRARY
+The 
+.I \\-profile=name
+argument allows you to specify an MPI profiling
+library to be used.  
+.I name
+can have two forms:
+
+.br
+A library in the same directory as the MPI library
+.br
+The name of a profile configuration file
+.br
+
+If 
+.I name
+is a library, then this library is included before the MPI
+library.  This allows the simple use of libraries that make use of the
+MPI profiling interface and that are installed in the same directory as
+the MPI library.
+
+If 
+.I name.conf
+is the name of a file in the sysconfdir directory, then this
+is read and may define the following variables:
+.PD 0
+.TP
+.B PROFILE_PRELIB 
+- Libraries (and paths) to include before the MPI library
+.PD 1
+.PD 0
+.TP
+.B PROFILE_POSTLIB 
+- Libraries to include after the MPI library
+.PD 1
+.PD 0
+.TP
+.B PROFILE_INCPATHS 
+- C preprocessor arguments for any include files
+For example, to add 
+.I /usr/local/myprof/include
+to the include path and
+the library 
+.I libmyprof.a
+in 
+.I /usr/local/myprof/lib
+to the link step,
+you could create the file 
+.I myprof.conf
+with the lines
+.PD 1
+
+.nf
+PROFILE_PRELIB="-L/usr/local/myprof/lib -lmyprof"
+PROFILE_INCPATHS="-I/usr/local/myprof/include"
+.fi
+
+and place it in the sysconfdir directory (this directory is set at
+configure time when MPICH is built).  Then using the command-line
+argument 
+.I \\-profile=myprof
+will cause these
+definitions to be added to the relevant compile commands.
+
+.SH SEE ALSO
+mpicc, mpicxx, mpifort, mpiexec
+.br

macx64/mpi/mpich/share/man/man3/MPIR_Type_commit.3

@@ -0,0 +1,11 @@
+.TH MPIR_Type_commit 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIR_Type_commit \-  nput Parameters: . datatype_p - pointer to MPI datatype 
+.SH SYNOPSIS
+.nf
+int MPIR_Type_commit(MPI_Datatype * datatype_p)
+.fi
+.SH OUTPUT PARAMETERS
+
+.SH RETURN VALUE
+0 on success, -1 on failure.

macx64/mpi/mpich/share/man/man3/MPIR_Type_contiguous.3

@@ -0,0 +1,28 @@
+.TH MPIR_Type_contiguous 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIR_Type_contiguous \-  create a contiguous datatype 
+.SH SYNOPSIS
+.nf
+int MPIR_Type_contiguous(int count, MPI_Datatype oldtype, MPI_Datatype * newtype)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B count 
+- number of elements in the contiguous block
+.PD 1
+.PD 0
+.TP
+.B oldtype 
+- type (using handle) of datatype on which vector is based
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newtype 
+- handle of new contiguous datatype
+.PD 1
+
+.SH RETURN VALUE
+MPI_SUCCESS on success, MPI error code on failure.

macx64/mpi/mpich/share/man/man3/MPIR_Type_dup.3

@@ -0,0 +1,27 @@
+.TH MPIR_Type_dup 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIR_Type_dup \-  create a copy of a datatype 
+.SH SYNOPSIS
+.nf
+#undef FUNCNAME
+#define FUNCNAME MPIR_Type_dup
+#undef FCNAME
+#define FCNAME MPL_QUOTE(FUNCNAME)
+int MPIR_Type_dup(MPI_Datatype oldtype, MPI_Datatype * newtype)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B oldtype 
+- handle of original datatype
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newtype 
+- handle of newly created copy of datatype
+.PD 1
+
+.SH RETURN VALUE
+0 on success, MPI error code on failure.

macx64/mpi/mpich/share/man/man3/MPIR_Type_get_contents.3

@@ -0,0 +1,51 @@
+.TH MPIR_Type_get_contents 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIR_Type_get_contents \-  get content information from datatype 
+.SH SYNOPSIS
+.nf
+int MPIR_Type_get_contents(MPI_Datatype datatype,
+                           int max_integers,
+                           int max_addresses,
+                           int max_datatypes,
+                           int array_of_integers[],
+                           MPI_Aint array_of_addresses[], MPI_Datatype array_of_datatypes[])
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B datatype 
+- MPI datatype
+.PD 1
+.PD 0
+.TP
+.B max_integers 
+- size of array_of_integers
+.PD 1
+.PD 0
+.TP
+.B max_addresses 
+- size of array_of_addresses
+.PD 1
+.PD 0
+.TP
+.B max_datatypes 
+- size of array_of_datatypes
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B array_of_integers 
+- integers used in creating type
+.PD 1
+.PD 0
+.TP
+.B array_of_addresses 
+- MPI_Aints used in creating type
+.PD 1
+.PD 0
+.TP
+.B array_of_datatypes 
+- MPI_Datatypes used in creating type
+.PD 1
+

macx64/mpi/mpich/share/man/man3/MPIR_Type_indexed.3

@@ -0,0 +1,49 @@
+.TH MPIR_Type_indexed 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIR_Type_indexed \-  create an indexed datatype 
+.SH SYNOPSIS
+.nf
+int MPIR_Type_indexed(int count,
+                      const int *blocklength_array,
+                      const void *displacement_array,
+                      int dispinbytes, MPI_Datatype oldtype, MPI_Datatype * newtype)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B count 
+- number of blocks in type
+.PD 1
+.PD 0
+.TP
+.B blocklength_array 
+- number of elements in each block
+.PD 1
+.PD 0
+.TP
+.B displacement_array 
+- offsets of blocks from start of type (see next
+parameter for units)
+.PD 1
+.PD 0
+.TP
+.B dispinbytes 
+- if nonzero, then displacements are in bytes (the
+displacement_array is an array of ints), otherwise they in terms of
+extent of oldtype (the displacement_array is an array of MPI_Aints)
+.PD 1
+.PD 0
+.TP
+.B oldtype 
+- type (using handle) of datatype on which new type is based
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newtype 
+- handle of new indexed datatype
+.PD 1
+
+.SH RETURN VALUE
+0 on success, -1 on failure.

macx64/mpi/mpich/share/man/man3/MPIR_Type_struct.3

@@ -0,0 +1,45 @@
+.TH MPIR_Type_struct 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIR_Type_struct \-  create a struct datatype 
+.SH SYNOPSIS
+.nf
+#undef FUNCNAME
+#define FUNCNAME MPIR_Type_struct
+#undef FCNAME
+#define FCNAME MPL_QUOTE(FUNCNAME)
+int MPIR_Type_struct(int count,
+                     const int *blocklength_array,
+                     const MPI_Aint * displacement_array,
+                     const MPI_Datatype * oldtype_array, MPI_Datatype * newtype)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B count 
+- number of blocks in vector
+.PD 1
+.PD 0
+.TP
+.B blocklength_array 
+- number of elements in each block
+.PD 1
+.PD 0
+.TP
+.B displacement_array 
+- offsets of blocks from start of type in bytes
+.PD 1
+.PD 0
+.TP
+.B oldtype_array 
+- types (using handle) of datatypes on which vector is based
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newtype 
+- handle of new struct datatype
+.PD 1
+
+.SH RETURN VALUE
+MPI_SUCCESS on success, MPI errno on failure.

macx64/mpi/mpich/share/man/man3/MPIR_Type_vector.3

@@ -0,0 +1,48 @@
+.TH MPIR_Type_vector 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIR_Type_vector \-  create a vector datatype 
+.SH SYNOPSIS
+.nf
+int MPIR_Type_vector(int count,
+                     int blocklength,
+                     MPI_Aint stride,
+                     int strideinbytes, MPI_Datatype oldtype, MPI_Datatype * newtype)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B count 
+- number of blocks in vector
+.PD 1
+.PD 0
+.TP
+.B blocklength 
+- number of elements in each block
+.PD 1
+.PD 0
+.TP
+.B stride 
+- distance from beginning of one block to the next (see next
+parameter for units)
+.PD 1
+.PD 0
+.TP
+.B strideinbytes 
+- if nonzero, then stride is in bytes, otherwise stride
+is in terms of extent of oldtype
+.PD 1
+.PD 0
+.TP
+.B oldtype 
+- type (using handle) of datatype on which vector is based
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newtype 
+- handle of new vector datatype
+.PD 1
+
+.SH RETURN VALUE
+0 on success, MPI error code on failure.

macx64/mpi/mpich/share/man/man3/MPIX_Comm_agree.3

@@ -0,0 +1,96 @@
+.TH MPIX_Comm_agree 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIX_Comm_agree \-  Performs agreement operation on comm 
+.SH SYNOPSIS
+.nf
+int MPIX_Comm_agree(MPI_Comm comm, int *flag)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- new communicator (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+

macx64/mpi/mpich/share/man/man3/MPIX_Comm_failure_ack.3

@@ -0,0 +1,96 @@
+.TH MPIX_Comm_failure_ack 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIX_Comm_failure_ack \-  Acknowledge the current group of failed processes 
+.SH SYNOPSIS
+.nf
+int MPIX_Comm_failure_ack(MPI_Comm comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- Communicator (handle)
+.PD 1
+
+.SH NOTES
+Because MPI specifies that null objects (e.g., 
+.I MPI_COMM_NULL
+) are invalid
+as input to MPI routines unless otherwise specified, using 
+.I MPI_COMM_NULL
+as input to this routine is an error.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPIX_Comm_failure_get_acked.3

@@ -0,0 +1,103 @@
+.TH MPIX_Comm_failure_get_acked 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIX_Comm_failure_get_acked \-  Get the group of acknowledged failures. 
+.SH SYNOPSIS
+.nf
+int MPIX_Comm_failure_get_acked(MPI_Comm comm, MPI_Group * failedgrp)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- Communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B failed_group 
+- Group (handle)
+.PD 1
+
+.SH NOTES
+Because MPI specifies that null objects (e.g., 
+.I MPI_COMM_NULL
+) are invalid
+as input to MPI routines unless otherwise specified, using 
+.I MPI_COMM_NULL
+as input to this routine is an error.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPIX_Comm_revoke.3

@@ -0,0 +1,79 @@
+.TH MPIX_Comm_revoke 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIX_Comm_revoke \-  Prevent a communicator from being used in the future 
+.SH SYNOPSIS
+.nf
+int MPIX_Comm_revoke(MPI_Comm comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator to revoke
+.PD 1
+
+.SH NOTES
+Asynchronously notifies all MPI processes associated with the communicator 
+.I comm
+\&.
+
+This will be manifest by returning the MPIX_ERR_REVOKED during a subsequent MPI
+call.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPIX_Comm_shrink.3

@@ -0,0 +1,96 @@
+.TH MPIX_Comm_shrink 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPIX_Comm_shrink \-  Creates a new communitor from an existing communicator while excluding failed processes 
+.SH SYNOPSIS
+.nf
+int MPIX_Comm_shrink(MPI_Comm comm, MPI_Comm * newcomm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- new communicator (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+

macx64/mpi/mpich/share/man/man3/MPI_Abort.3

@@ -0,0 +1,104 @@
+.TH MPI_Abort 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Abort \-  Terminates MPI execution environment 
+.SH SYNOPSIS
+.nf
+int MPI_Abort(MPI_Comm comm, int errorcode)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator of tasks to abort
+.PD 1
+.PD 0
+.TP
+.B errorcode 
+- error code to return to invoking environment
+.PD 1
+
+.SH NOTES
+Terminates all MPI processes associated with the communicator 
+.I comm
+; in
+most systems (all to date), terminates 
+.B all
+processes.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+The user is responsible for ensuring that multiple threads do not try to
+update the same MPI object from different threads.  This routine should
+not be used from within a signal handler.
+
+The MPI standard defined a thread-safe interface but this does not
+mean that all routines may be called without any thread locks.  For
+example, two threads must not attempt to change the contents of the
+same 
+.I MPI_Info
+object concurrently.  The user is responsible in this
+case for using some mechanism, such as thread locks, to ensure that
+only one thread at a time makes use of this routine.
+Because the 
+.I MPI_Abort
+routine is intended to ensure that an MPI
+process (and possibly an entire job), it cannot wait for a thread to
+release a lock or other mechanism for atomic access.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Accumulate.3

@@ -0,0 +1,171 @@
+.TH MPI_Accumulate 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Accumulate \-  Accumulate data into the target process using remote memory access 
+.SH SYNOPSIS
+.nf
+int MPI_Accumulate(const void *origin_addr, int origin_count, MPI_Datatype
+                   origin_datatype, int target_rank, MPI_Aint
+                   target_disp, int target_count, MPI_Datatype
+                   target_datatype, MPI_Op op, MPI_Win win)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B origin_addr 
+- initial address of buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B origin_count 
+- number of entries in buffer (nonnegative integer)
+.PD 1
+.PD 0
+.TP
+.B origin_datatype 
+- datatype of each buffer entry (handle)
+.PD 1
+.PD 0
+.TP
+.B target_rank 
+- rank of target (nonnegative integer)
+.PD 1
+.PD 0
+.TP
+.B target_disp 
+- displacement from start of window to beginning of target
+buffer (nonnegative integer)
+.PD 1
+.PD 0
+.TP
+.B target_count 
+- number of entries in target buffer (nonnegative integer)
+.PD 1
+.PD 0
+.TP
+.B target_datatype 
+- datatype of each entry in target buffer (handle)
+.PD 1
+.PD 0
+.TP
+.B op 
+- predefined reduce operation (handle)
+.PD 1
+.PD 0
+.TP
+.B win 
+- window object (handle)
+.PD 1
+
+.SH NOTES
+The basic components of both the origin and target datatype must be the same
+predefined datatype (e.g., all 
+.I MPI_INT
+or all 
+.I MPI_DOUBLE_PRECISION
+).
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COUNT 
+- Invalid count argument.  Count arguments must be 
+non-negative; a count of zero is often valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_RANK 
+- Invalid source or destination rank.  Ranks must be between
+zero and the size of the communicator minus one; ranks in a receive
+(
+.I MPI_Recv
+, 
+.I MPI_Irecv
+, 
+.I MPI_Sendrecv
+, etc.) may also be 
+.I MPI_ANY_SOURCE
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TYPE 
+- Invalid datatype argument.  Additionally, this error can
+occur if an uncommitted MPI_Datatype (see 
+.I MPI_Type_commit
+) is used
+in a communication call.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_WIN 
+- Invalid MPI window object
+.PD 1
+
+.SH SEE ALSO
+MPI_Raccumulate
+.br

macx64/mpi/mpich/share/man/man3/MPI_Add_error_class.3

@@ -0,0 +1,88 @@
+.TH MPI_Add_error_class 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Add_error_class \-  Add an MPI error class to the known classes 
+.SH SYNOPSIS
+.nf
+int MPI_Add_error_class(int *errorclass)
+.fi
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B errorclass 
+- New error class
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_OTHER 
+- Other error; use 
+.I MPI_Error_string
+to get more information
+about this error code. 
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Add_error_code.3

@@ -0,0 +1,95 @@
+.TH MPI_Add_error_code 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Add_error_code \-  Add an MPI error code to an MPI error class 
+.SH SYNOPSIS
+.nf
+int MPI_Add_error_code(int errorclass, int *errorcode)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B errorclass 
+- Error class to add an error code.
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B errorcode 
+- New error code for this error class.
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_OTHER 
+- Other error; use 
+.I MPI_Error_string
+to get more information
+about this error code. 
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Add_error_string.3

@@ -0,0 +1,105 @@
+.TH MPI_Add_error_string 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Add_error_string \-  Associates an error string with an MPI error code or class 
+.SH SYNOPSIS
+.nf
+int MPI_Add_error_string(int errorcode, const char *string)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B errorcode 
+- error code or class (integer)
+.PD 1
+.PD 0
+.TP
+.B string 
+- text corresponding to errorcode (string)
+.PD 1
+
+.SH NOTES
+The string must be no more than 
+.I MPI_MAX_ERROR_STRING
+characters long.
+The length of the string is as defined in the calling language.
+The length of the string does not include the null terminator in C or C++.
+Note that the string is 
+.I const
+even though the MPI standard does not
+specify it that way.
+
+According to the MPI-2 standard, it is erroneous to call 
+.I MPI_Add_error_string
+for an error code or class with a value less than or equal
+to 
+.I MPI_ERR_LASTCODE
+\&.
+Thus, you cannot replace the predefined error messages
+with this routine.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Address.3

@@ -0,0 +1,111 @@
+.TH MPI_Address 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Address \-  Gets the address of a location in memory 
+.SH SYNOPSIS
+.nf
+int MPI_Address(void *location, MPI_Aint * address)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B location 
+- location in caller memory (choice)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B address 
+- address of location (address integer)
+.PD 1
+
+.SH NOTE
+This routine is provided for both the Fortran and C programmers.
+On many systems, the address returned by this routine will be the same
+as produced by the C 
+.I &
+operator, but this is not required in C and
+may not be true of systems with word- rather than byte-oriented
+instructions or systems with segmented address spaces.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is both thread- and interrupt-safe.
+This means that this routine may safely be used by multiple threads and
+from within a signal handler.
+
+.SH DEPRECATED FUNCTION
+The MPI-2 standard deprecated a number of routines because MPI-2 provides
+better versions.  This routine is one of those that was deprecated.  The
+routine may continue to be used, but new code should use the replacement
+routine.
+The replacement for this routine is 
+.I MPI_Get_address
+\&.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_OTHER 
+- Other error; use 
+.I MPI_Error_string
+to get more information
+about this error code. 
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Aint_add.3

@@ -0,0 +1,38 @@
+.TH MPI_Aint_add 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Aint_add \-  Returns the sum of base and disp 
+.SH SYNOPSIS
+.nf
+MPI_Aint MPI_Aint_add(MPI_Aint base, MPI_Aint disp)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B base 
+- base address (integer)
+.PD 1
+.PD 0
+.TP
+.B disp 
+- displacement (integer)
+.PD 1
+
+.SH RETURN VALUE
+Sum of the base and disp argument
+
+.SH NOTES
+MPI_Aint_Add produces a new MPI_Aint value that is equivalent to the sum of the
+base and disp arguments, where base represents a base address returned by a call
+to MPI_GET_ADDRESS and disp represents a signed integer displacement. The resulting
+address is valid only at the process that generated base, and it must correspond
+to a location in the same object referenced by base. The addition is performed in
+a manner that results in the correct MPI_Aint representation of the output address,
+as if the process that originally produced base had called:
+.nf
+MPI_Get_address((char *) base + disp, &result)
+.fi
+
+
+.SH SEE ALSO
+MPI_Aint_diff
+.br

macx64/mpi/mpich/share/man/man3/MPI_Aint_diff.3

@@ -0,0 +1,39 @@
+.TH MPI_Aint_diff 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Aint_diff \-  Returns the difference between addr1 and addr2 
+.SH SYNOPSIS
+.nf
+MPI_Aint MPI_Aint_diff(MPI_Aint addr1, MPI_Aint addr2)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B addr1 
+- minuend address (integer)
+.PD 1
+.PD 0
+.TP
+.B addr2 
+- subtrahend address (integer)
+.PD 1
+
+.SH RETURN VALUE
+Difference between addr1 and addr2
+
+.SH NOTES
+MPI_Aint_diff produces a new MPI_Aint value that is equivalent to the difference
+between addr1 and addr2 arguments, where addr1 and addr2 represent addresses
+returned by calls to MPI_GET_ADDRESS. The resulting address is valid only at the
+process that generated addr1 and addr2, and addr1 and addr2 must correspond to
+locations in the same object in the same process. The difference is calculated
+in a manner that results the signed difference from addr1 to addr2, as if the
+process that originally produced the addresses had called
+.nf
+(char *) addr1 - (char *) addr2
+.fi
+
+on the addresses initially passed to MPI_GET_ADDRESS.
+
+.SH SEE ALSO
+MPI_Aint_add
+.br

macx64/mpi/mpich/share/man/man3/MPI_Allgather.3

@@ -0,0 +1,169 @@
+.TH MPI_Allgather 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Allgather \-  Gathers data from all tasks and distribute the combined data to all tasks 
+.SH SYNOPSIS
+.nf
+int MPI_Allgather(const void *sendbuf, int sendcount, MPI_Datatype sendtype,
+                  void *recvbuf, int recvcount, MPI_Datatype recvtype, MPI_Comm comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B sendbuf 
+- starting address of send buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B sendcount 
+- number of elements in send buffer (integer)
+.PD 1
+.PD 0
+.TP
+.B sendtype 
+- data type of send buffer elements (handle)
+.PD 1
+.PD 0
+.TP
+.B recvcount 
+- number of elements received from any process (integer)
+.PD 1
+.PD 0
+.TP
+.B recvtype 
+- data type of receive buffer elements (handle)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B recvbuf 
+- address of receive buffer (choice)
+.PD 1
+
+.SH NOTES
+The MPI standard (1.0 and 1.1) says that
+.br
+
+.br
+
+The jth block of data sent from  each process is received by every process
+and placed in the jth block of the buffer 
+.I recvbuf
+\&.
+
+.br
+
+.br
+
+This is misleading; a better description is
+.br
+
+.br
+
+The block of data sent from the jth process is received by every
+process and placed in the jth block of the buffer 
+.I recvbuf
+\&.
+
+.br
+
+.br
+
+This text was suggested by Rajeev Thakur and has been adopted as a
+clarification by the MPI Forum.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COUNT 
+- Invalid count argument.  Count arguments must be 
+non-negative; a count of zero is often valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TYPE 
+- Invalid datatype argument.  Additionally, this error can
+occur if an uncommitted MPI_Datatype (see 
+.I MPI_Type_commit
+) is used
+in a communication call.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_BUFFER 
+- Invalid buffer pointer.  Usually a null buffer where
+one is not valid.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Allgatherv.3

@@ -0,0 +1,174 @@
+.TH MPI_Allgatherv 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Allgatherv \-  Gathers data from all tasks and deliver the combined data to all tasks 
+.SH SYNOPSIS
+.nf
+int MPI_Allgatherv(const void *sendbuf, int sendcount, MPI_Datatype sendtype,
+                   void *recvbuf, const int *recvcounts, const int *displs,
+                   MPI_Datatype recvtype, MPI_Comm comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B sendbuf 
+- starting address of send buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B sendcount 
+- number of elements in send buffer (integer)
+.PD 1
+.PD 0
+.TP
+.B sendtype 
+- data type of send buffer elements (handle)
+.PD 1
+.PD 0
+.TP
+.B recvcounts 
+- integer array (of length group size)
+containing the number of elements that are to be received from each process
+.PD 1
+.PD 0
+.TP
+.B displs 
+- integer array (of length group size). Entry
+.I i
+specifies the displacement (relative to recvbuf) at
+which to place the incoming data from process  
+.I i
+
+.PD 1
+.PD 0
+.TP
+.B recvtype 
+- data type of receive buffer elements (handle)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B recvbuf 
+- address of receive buffer (choice)
+.PD 1
+
+.SH NOTES
+The MPI standard (1.0 and 1.1) says that
+.br
+
+.br
+
+The jth block of data sent from
+each process is received by every process and placed in the jth block of the
+buffer 
+.I recvbuf
+\&.
+
+.br
+
+.br
+
+This is misleading; a better description is
+.br
+
+.br
+
+The block of data sent from the jth process is received by every
+process and placed in the jth block of the buffer 
+.I recvbuf
+\&.
+
+.br
+
+.br
+
+This text was suggested by Rajeev Thakur, and has been adopted as a
+clarification to the MPI standard by the MPI-Forum.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_ERR_BUFFER 
+- Invalid buffer pointer.  Usually a null buffer where
+one is not valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COUNT 
+- Invalid count argument.  Count arguments must be 
+non-negative; a count of zero is often valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TYPE 
+- Invalid datatype argument.  Additionally, this error can
+occur if an uncommitted MPI_Datatype (see 
+.I MPI_Type_commit
+) is used
+in a communication call.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Alloc_mem.3

@@ -0,0 +1,128 @@
+.TH MPI_Alloc_mem 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Alloc_mem \-  Allocate memory for message passing and RMA 
+.SH SYNOPSIS
+.nf
+int MPI_Alloc_mem(MPI_Aint size, MPI_Info info, void *baseptr)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B size 
+- size of memory segment in bytes (nonnegative integer)
+.PD 1
+.PD 0
+.TP
+.B info 
+- info argument (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B baseptr 
+- pointer to beginning of memory segment allocated
+.PD 1
+
+.SH NOTES
+Using this routine from Fortran requires that the Fortran compiler accept
+a common pointer extension.  See Section 4.11 (Memory Allocation) in the
+MPI-2 standard for more information and examples.
+
+Also note that while 
+.I baseptr
+is a 
+.I void *
+type, this is
+simply to allow easy use of any pointer object for this parameter.
+In fact, this argument is really a 
+.I void **
+type, that is, a
+pointer to a pointer.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_INFO 
+- Invalid Info 
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_NO_MEM 
+- Insufficient memory available for allocation by 
+.I MPI_Alloc_mem
+
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Allreduce.3

@@ -0,0 +1,163 @@
+.TH MPI_Allreduce 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Allreduce \-  Combines values from all processes and distributes the result back to all processes 
+.SH SYNOPSIS
+.nf
+int MPI_Allreduce(const void *sendbuf, void *recvbuf, int count,
+                  MPI_Datatype datatype, MPI_Op op, MPI_Comm comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B sendbuf 
+- starting address of send buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B count 
+- number of elements in send buffer (integer)
+.PD 1
+.PD 0
+.TP
+.B datatype 
+- data type of elements of send buffer (handle)
+.PD 1
+.PD 0
+.TP
+.B op 
+- operation (handle)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B recvbuf 
+- starting address of receive buffer (choice)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH NOTES ON COLLECTIVE OPERATIONS
+
+The reduction functions (
+.I MPI_Op
+) do not return an error value.  As a result,
+if the functions detect an error, all they can do is either call 
+.I MPI_Abort
+or silently skip the problem.  Thus, if you change the error handler from
+.I MPI_ERRORS_ARE_FATAL
+to something else, for example, 
+.I MPI_ERRORS_RETURN
+,
+then no error may be indicated.
+
+The reason for this is the performance problems in ensuring that
+all collective routines return the same error value.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_ERR_BUFFER 
+- Invalid buffer pointer.  Usually a null buffer where
+one is not valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COUNT 
+- Invalid count argument.  Count arguments must be 
+non-negative; a count of zero is often valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TYPE 
+- Invalid datatype argument.  Additionally, this error can
+occur if an uncommitted MPI_Datatype (see 
+.I MPI_Type_commit
+) is used
+in a communication call.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_OP 
+- Invalid operation.  MPI operations (objects of type 
+.I MPI_Op
+)
+must either be one of the predefined operations (e.g., 
+.I MPI_SUM
+) or
+created with 
+.I MPI_Op_create
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Alltoall.3

@@ -0,0 +1,137 @@
+.TH MPI_Alltoall 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Alltoall \-  Sends data from all to all processes 
+.SH SYNOPSIS
+.nf
+int MPI_Alltoall(const void *sendbuf, int sendcount, MPI_Datatype sendtype,
+                 void *recvbuf, int recvcount, MPI_Datatype recvtype, MPI_Comm comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B sendbuf 
+- starting address of send buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B sendcount 
+- number of elements to send to each process (integer)
+.PD 1
+.PD 0
+.TP
+.B sendtype 
+- data type of send buffer elements (handle)
+.PD 1
+.PD 0
+.TP
+.B recvcount 
+- number of elements received from any process (integer)
+.PD 1
+.PD 0
+.TP
+.B recvtype 
+- data type of receive buffer elements (handle)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B recvbuf 
+- address of receive buffer (choice)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COUNT 
+- Invalid count argument.  Count arguments must be 
+non-negative; a count of zero is often valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TYPE 
+- Invalid datatype argument.  Additionally, this error can
+occur if an uncommitted MPI_Datatype (see 
+.I MPI_Type_commit
+) is used
+in a communication call.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_BUFFER 
+- Invalid buffer pointer.  Usually a null buffer where
+one is not valid.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Alltoallv.3

@@ -0,0 +1,161 @@
+.TH MPI_Alltoallv 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Alltoallv \-  Sends data from all to all processes; each process may send a different amount of data and provide displacements for the input and output data. 
+.SH SYNOPSIS
+.nf
+int MPI_Alltoallv(const void *sendbuf, const int *sendcounts,
+                  const int *sdispls, MPI_Datatype sendtype, void *recvbuf,
+                  const int *recvcounts, const int *rdispls, MPI_Datatype recvtype, MPI_Comm comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B sendbuf 
+- starting address of send buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B sendcounts 
+- integer array equal to the group size
+specifying the number of elements to send to each processor
+.PD 1
+.PD 0
+.TP
+.B sdispls 
+- integer array (of length group size). Entry
+.I j
+specifies the displacement (relative to sendbuf  from
+which to take the outgoing data destined for process  
+.I j
+
+.PD 1
+.PD 0
+.TP
+.B sendtype 
+- data type of send buffer elements (handle)
+.PD 1
+.PD 0
+.TP
+.B recvcounts 
+- integer array equal to the group size
+specifying the maximum number of elements that can be received from
+each processor
+.PD 1
+.PD 0
+.TP
+.B rdispls 
+- integer array (of length group size). Entry
+.I i
+specifies the displacement (relative to recvbuf  at
+which to place the incoming data from process  
+.I i
+
+.PD 1
+.PD 0
+.TP
+.B recvtype 
+- data type of receive buffer elements (handle)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B recvbuf 
+- address of receive buffer (choice)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COUNT 
+- Invalid count argument.  Count arguments must be 
+non-negative; a count of zero is often valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TYPE 
+- Invalid datatype argument.  Additionally, this error can
+occur if an uncommitted MPI_Datatype (see 
+.I MPI_Type_commit
+) is used
+in a communication call.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_BUFFER 
+- Invalid buffer pointer.  Usually a null buffer where
+one is not valid.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Alltoallw.3

@@ -0,0 +1,164 @@
+.TH MPI_Alltoallw 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Alltoallw \-  Generalized all-to-all communication allowing different datatypes, counts, and displacements for each partner 
+.SH SYNOPSIS
+.nf
+int MPI_Alltoallw(const void *sendbuf, const int sendcounts[],
+                  const int sdispls[], const MPI_Datatype sendtypes[],
+                  void *recvbuf, const int recvcounts[], const int rdispls[],
+                  const MPI_Datatype recvtypes[], MPI_Comm comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B sendbuf 
+- starting address of send buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B sendcounts 
+- integer array equal to the group size specifying the number of
+elements to send to each processor (integer)
+.PD 1
+.PD 0
+.TP
+.B sdispls 
+- integer array (of length group size). Entry j specifies the
+displacement in bytes (relative to sendbuf) from which to take the outgoing
+data destined for process j
+.PD 1
+.PD 0
+.TP
+.B sendtypes 
+- array of datatypes (of length group size). Entry j specifies the
+type of data to send to process j (handle)
+.PD 1
+.PD 0
+.TP
+.B recvcounts 
+- integer array equal to the group size specifying the number of
+elements that can be received from each processor (integer)
+.PD 1
+.PD 0
+.TP
+.B rdispls 
+- integer array (of length group size). Entry i specifies the
+displacement in bytes (relative to recvbuf) at which to place the incoming
+data from process i
+.PD 1
+.PD 0
+.TP
+.B recvtypes 
+- array of datatypes (of length group size). Entry i specifies
+the type of data received from process i (handle)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B recvbuf 
+- address of receive buffer (choice)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COUNT 
+- Invalid count argument.  Count arguments must be 
+non-negative; a count of zero is often valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TYPE 
+- Invalid datatype argument.  Additionally, this error can
+occur if an uncommitted MPI_Datatype (see 
+.I MPI_Type_commit
+) is used
+in a communication call.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Attr_delete.3

@@ -0,0 +1,109 @@
+.TH MPI_Attr_delete 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Attr_delete \-  Deletes an attribute value associated with a key on a communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Attr_delete(MPI_Comm comm, int keyval)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator to which attribute is attached (handle)
+.PD 1
+.PD 0
+.TP
+.B keyval 
+- The key value of the deleted attribute (integer)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH DEPRECATED FUNCTION
+The MPI-2 standard deprecated a number of routines because MPI-2 provides
+better versions.  This routine is one of those that was deprecated.  The
+routine may continue to be used, but new code should use the replacement
+routine.
+The replacement for this routine is 
+.I MPI_Comm_delete_attr
+\&.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- This error class is associated with an error code that 
+indicates that an attempt was made to free one of the permanent keys.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Attr_get.3

@@ -0,0 +1,157 @@
+.TH MPI_Attr_get 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Attr_get \-  Retrieves attribute value by key 
+.SH SYNOPSIS
+.nf
+int MPI_Attr_get(MPI_Comm comm, int keyval, void *attribute_val, int *flag)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator to which attribute is attached (handle)
+.PD 1
+.PD 0
+.TP
+.B keyval 
+- key value (integer)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B attribute_val 
+- attribute value, unless 
+.I flag
+= false
+.PD 1
+.PD 0
+.TP
+.B flag 
+- true if an attribute value was extracted;  false if no attribute is
+associated with the key
+.PD 1
+
+.SH NOTES
+Attributes must be extracted from the same language as they were inserted
+in with 
+.I MPI_ATTR_PUT
+\&.
+The notes for C and Fortran below explain why.
+
+.SH NOTES FOR C
+Even though the 
+.I attribute_val
+argument is declared as 
+.I void *
+, it is
+really the address of a void pointer (i.e., a 
+.I void **
+).  Using
+a 
+.I void *
+, however, is more in keeping with C idiom and allows the
+pointer to be passed without additional casts.
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH DEPRECATED FUNCTION
+The MPI-2 standard deprecated a number of routines because MPI-2 provides
+better versions.  This routine is one of those that was deprecated.  The
+routine may continue to be used, but new code should use the replacement
+routine.
+The replacement for this routine is 
+.I MPI_Comm_get_attr
+\&.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+The 
+.I attribute_val
+in Fortran is a pointer to a Fortran integer, not
+a pointer to a 
+.I void *
+\&.
+
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_KEYVAL 
+- Invalid keyval
+.PD 1
+
+.SH SEE ALSO
+MPI_Attr_put, MPI_Keyval_create, MPI_Attr_delete, MPI_Comm_get_attr
+.br

macx64/mpi/mpich/share/man/man3/MPI_Attr_put.3

@@ -0,0 +1,159 @@
+.TH MPI_Attr_put 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Attr_put \-  Stores attribute value associated with a key 
+.SH SYNOPSIS
+.nf
+int MPI_Attr_put(MPI_Comm comm, int keyval, void *attribute_val)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator to which attribute will be attached (handle)
+.PD 1
+.PD 0
+.TP
+.B keyval 
+- key value, as returned by  
+.I MPI_KEYVAL_CREATE
+(integer)
+.PD 1
+.PD 0
+.TP
+.B attribute_val 
+- attribute value
+.PD 1
+
+.SH NOTES
+Values of the permanent attributes 
+.I MPI_TAG_UB
+, 
+.I MPI_HOST
+, 
+.I MPI_IO
+,
+.I MPI_WTIME_IS_GLOBAL
+, 
+.I MPI_UNIVERSE_SIZE
+, 
+.I MPI_LASTUSEDCODE
+, and
+.I MPI_APPNUM
+may not be changed.
+
+The type of the attribute value depends on whether C, C++, or Fortran
+is being used.
+In C and C++, an attribute value is a pointer (
+.I void *
+); in Fortran,
+it is a single
+integer (
+.B not
+a pointer, since Fortran has no pointers and there are systems
+for which a pointer does not fit in an integer (e.g., any > 32 bit address
+system that uses 64 bits for Fortran 
+.I DOUBLE PRECISION
+).
+
+If an attribute is already present, the delete function (specified when the
+corresponding keyval was created) will be called.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH DEPRECATED FUNCTION
+The MPI-2 standard deprecated a number of routines because MPI-2 provides
+better versions.  This routine is one of those that was deprecated.  The
+routine may continue to be used, but new code should use the replacement
+routine.
+The replacement for this routine is 
+.I MPI_Comm_set_attr
+\&.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_KEYVAL 
+- Invalid keyval
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- This error class is associated with an error code that 
+indicates that an attempt was made to free one of the permanent keys.
+.PD 1
+
+.SH SEE ALSO
+MPI_Attr_get, MPI_Keyval_create, MPI_Attr_delete, MPI_Comm_set_attr
+.br

macx64/mpi/mpich/share/man/man3/MPI_Barrier.3

@@ -0,0 +1,93 @@
+.TH MPI_Barrier 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Barrier \-  Blocks until all processes in the communicator have reached this routine. 
+.SH SYNOPSIS
+.nf
+int MPI_Barrier(MPI_Comm comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH NOTES
+Blocks the caller until all processes in the communicator have called it;
+that is, the call returns at any process only after all members of the
+communicator have entered the call.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Bcast.3

@@ -0,0 +1,138 @@
+.TH MPI_Bcast 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Bcast \-  Broadcasts a message from the process with rank "root" to all other processes of the communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Bcast(void *buffer, int count, MPI_Datatype datatype, int root, MPI_Comm comm)
+.fi
+.SH INPUT/OUTPUT PARAMETERS
+.PD 0
+.TP
+.B buffer 
+- starting address of buffer (choice)
+.PD 1
+
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B count 
+- number of entries in buffer (integer)
+.PD 1
+.PD 0
+.TP
+.B datatype 
+- data type of buffer (handle)
+.PD 1
+.PD 0
+.TP
+.B root 
+- rank of broadcast root (integer)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COUNT 
+- Invalid count argument.  Count arguments must be 
+non-negative; a count of zero is often valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TYPE 
+- Invalid datatype argument.  Additionally, this error can
+occur if an uncommitted MPI_Datatype (see 
+.I MPI_Type_commit
+) is used
+in a communication call.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_BUFFER 
+- Invalid buffer pointer.  Usually a null buffer where
+one is not valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ROOT 
+- Invalid root.  The root must be specified as a rank in the
+communicator.  Ranks must be between zero and the size of the communicator 
+minus one.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Bsend.3

@@ -0,0 +1,209 @@
+.TH MPI_Bsend 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Bsend \-  Basic send with user-provided buffering 
+.SH SYNOPSIS
+.nf
+int MPI_Bsend(const void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B buf 
+- initial address of send buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B count 
+- number of elements in send buffer (nonnegative integer)
+.PD 1
+.PD 0
+.TP
+.B datatype 
+- datatype of each send buffer element (handle)
+.PD 1
+.PD 0
+.TP
+.B dest 
+- rank of destination (integer)
+.PD 1
+.PD 0
+.TP
+.B tag 
+- message tag (integer)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH NOTES
+This send is provided as a convenience function; it allows the user to
+send messages without worring about where they are buffered (because the
+user 
+.B must
+have provided buffer space with 
+.I MPI_Buffer_attach
+).
+
+In deciding how much buffer space to allocate, remember that the buffer space
+is not available for reuse by subsequent 
+.I MPI_Bsend
+s unless you are certain
+that the message
+has been received (not just that it should have been received).  For example,
+this code does not allocate enough buffer space
+.nf
+MPI_Buffer_attach(b, n*sizeof(double) + MPI_BSEND_OVERHEAD);
+for (i=0; i<m; i++) {
+MPI_Bsend(buf, n, MPI_DOUBLE, ...);
+}
+.fi
+
+because only enough buffer space is provided for a single send, and the
+loop may start a second 
+.I MPI_Bsend
+before the first is done making use of the
+buffer.
+
+In C, you can
+force the messages to be delivered by
+.nf
+MPI_Buffer_detach(&b, &n);
+MPI_Buffer_attach(b, n);
+.fi
+
+(The 
+.I MPI_Buffer_detach
+will not complete until all buffered messages are
+delivered.)
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COUNT 
+- Invalid count argument.  Count arguments must be 
+non-negative; a count of zero is often valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TYPE 
+- Invalid datatype argument.  Additionally, this error can
+occur if an uncommitted MPI_Datatype (see 
+.I MPI_Type_commit
+) is used
+in a communication call.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_RANK 
+- Invalid source or destination rank.  Ranks must be between
+zero and the size of the communicator minus one; ranks in a receive
+(
+.I MPI_Recv
+, 
+.I MPI_Irecv
+, 
+.I MPI_Sendrecv
+, etc.) may also be 
+.I MPI_ANY_SOURCE
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TAG 
+- Invalid tag argument.  Tags must be non-negative; tags
+in a receive (
+.I MPI_Recv
+, 
+.I MPI_Irecv
+, 
+.I MPI_Sendrecv
+, etc.) may
+also be 
+.I MPI_ANY_TAG
+\&.
+The largest tag value is available through the 
+the attribute 
+.I MPI_TAG_UB
+\&.
+
+.PD 1
+
+.SH SEE ALSO
+MPI_Buffer_attach, MPI_Ibsend, MPI_Bsend_init
+.br

macx64/mpi/mpich/share/man/man3/MPI_Bsend_init.3

@@ -0,0 +1,176 @@
+.TH MPI_Bsend_init 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Bsend_init \-  Builds a handle for a buffered send 
+.SH SYNOPSIS
+.nf
+int MPI_Bsend_init(const void *buf, int count, MPI_Datatype datatype,
+                   int dest, int tag, MPI_Comm comm, MPI_Request * request)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B buf 
+- initial address of send buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B count 
+- number of elements sent (integer)
+.PD 1
+.PD 0
+.TP
+.B datatype 
+- type of each element (handle)
+.PD 1
+.PD 0
+.TP
+.B dest 
+- rank of destination (integer)
+.PD 1
+.PD 0
+.TP
+.B tag 
+- message tag (integer)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B request 
+- communication request (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COUNT 
+- Invalid count argument.  Count arguments must be 
+non-negative; a count of zero is often valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TYPE 
+- Invalid datatype argument.  Additionally, this error can
+occur if an uncommitted MPI_Datatype (see 
+.I MPI_Type_commit
+) is used
+in a communication call.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_RANK 
+- Invalid source or destination rank.  Ranks must be between
+zero and the size of the communicator minus one; ranks in a receive
+(
+.I MPI_Recv
+, 
+.I MPI_Irecv
+, 
+.I MPI_Sendrecv
+, etc.) may also be 
+.I MPI_ANY_SOURCE
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TAG 
+- Invalid tag argument.  Tags must be non-negative; tags
+in a receive (
+.I MPI_Recv
+, 
+.I MPI_Irecv
+, 
+.I MPI_Sendrecv
+, etc.) may
+also be 
+.I MPI_ANY_TAG
+\&.
+The largest tag value is available through the 
+the attribute 
+.I MPI_TAG_UB
+\&.
+
+.PD 1
+
+.SH SEE ALSO
+MPI_Buffer_attach
+.br

macx64/mpi/mpich/share/man/man3/MPI_Buffer_attach.3

@@ -0,0 +1,159 @@
+.TH MPI_Buffer_attach 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Buffer_attach \-  Attaches a user-provided buffer for sending 
+.SH SYNOPSIS
+.nf
+int MPI_Buffer_attach(void *buffer, int size)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B buffer 
+- initial buffer address (choice)
+.PD 1
+.PD 0
+.TP
+.B size 
+- buffer size, in bytes (integer)
+.PD 1
+
+.SH NOTES
+The size given should be the sum of the sizes of all outstanding Bsends that
+you intend to have, plus 
+.I MPI_BSEND_OVERHEAD
+for each Bsend that you do.
+For the purposes of calculating size, you should use 
+.I MPI_Pack_size
+\&.
+
+In other words, in the code
+.nf
+MPI_Buffer_attach(buffer, size);
+MPI_Bsend(..., count=20, datatype=type1,  ...);
+\&.
+\&.
+\&.
+MPI_Bsend(..., count=40, datatype=type2, ...);
+.fi
+
+the value of 
+.I size
+in the 
+.I MPI_Buffer_attach
+call should be greater than
+the value computed by
+.nf
+MPI_Pack_size(20, type1, comm, &s1);
+MPI_Pack_size(40, type2, comm, &s2);
+size = s1 + s2 + 2 * MPI_BSEND_OVERHEAD;
+.fi
+
+The 
+.I MPI_BSEND_OVERHEAD
+gives the maximum amount of space that may be used in
+the buffer for use by the BSEND routines in using the buffer.  This value
+is in 
+.I mpi.h
+(for C) and 
+.I mpif.h
+(for Fortran).
+
+.SH THREAD AND INTERRUPT SAFETY
+
+The user is responsible for ensuring that multiple threads do not try to
+update the same MPI object from different threads.  This routine should
+not be used from within a signal handler.
+
+The MPI standard defined a thread-safe interface but this does not
+mean that all routines may be called without any thread locks.  For
+example, two threads must not attempt to change the contents of the
+same 
+.I MPI_Info
+object concurrently.  The user is responsible in this
+case for using some mechanism, such as thread locks, to ensure that
+only one thread at a time makes use of this routine.
+Because the buffer for buffered sends (e.g., 
+.I MPI_Bsend
+) is shared by all
+threads in a process, the user is responsible for ensuring that only
+one thread at a time calls this routine or 
+.I MPI_Buffer_detach
+\&.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_BUFFER 
+- Invalid buffer pointer.  Usually a null buffer where
+one is not valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_INTERN 
+- An internal error has been detected.  This is fatal.
+Please send a bug report to 
+.I mpi-bugs@mcs.anl.gov
+\&.
+
+.PD 1
+
+.SH SEE ALSO
+MPI_Buffer_detach, MPI_Bsend
+.br

macx64/mpi/mpich/share/man/man3/MPI_Buffer_detach.3

@@ -0,0 +1,130 @@
+.TH MPI_Buffer_detach 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Buffer_detach \-  Removes an existing buffer (for use in MPI_Bsend etc) 
+.SH SYNOPSIS
+.nf
+int MPI_Buffer_detach(void *buffer_addr, int *size)
+.fi
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B buffer_addr 
+- initial buffer address (choice)
+.PD 1
+.PD 0
+.TP
+.B size 
+- buffer size, in bytes (integer)
+.PD 1
+
+.SH NOTES
+The reason that 
+.I MPI_Buffer_detach
+returns the address and size of the
+buffer being detached is to allow nested libraries to replace and restore
+the buffer.  For example, consider
+
+.nf
+int size, mysize, idummy;
+void *ptr, *myptr, *dummy;
+MPI_Buffer_detach(&ptr, &size);
+MPI_Buffer_attach(myptr, mysize);
+\&.
+\&.
+\&.
+\&.
+\&.
+\&.
+library code ...
+\&.
+\&.
+\&.
+MPI_Buffer_detach(&dummy, &idummy);
+MPI_Buffer_attach(ptr, size);
+.fi
+
+
+This is much like the action of the Unix signal routine and has the same
+strengths (it is simple) and weaknesses (it only works for nested usages).
+
+Note that for this approach to work, MPI_Buffer_detach must return MPI_SUCCESS
+even when there is no buffer to detach.  In that case, it returns a size of
+zero.  The MPI 1.1 standard for 
+.I MPI_BUFFER_DETACH
+contains the text
+
+.nf
+The statements made in this section describe the behavior of MPI for
+buffered-mode sends. When no buffer is currently associated, MPI behaves
+as if a zero-sized buffer is associated with the process.
+.fi
+
+
+This could be read as applying only to the various Bsend routines.  This
+implementation takes the position that this applies to 
+.I MPI_BUFFER_DETACH
+as well.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+The user is responsible for ensuring that multiple threads do not try to
+update the same MPI object from different threads.  This routine should
+not be used from within a signal handler.
+
+The MPI standard defined a thread-safe interface but this does not
+mean that all routines may be called without any thread locks.  For
+example, two threads must not attempt to change the contents of the
+same 
+.I MPI_Info
+object concurrently.  The user is responsible in this
+case for using some mechanism, such as thread locks, to ensure that
+only one thread at a time makes use of this routine.
+Because the buffer for buffered sends (e.g., 
+.I MPI_Bsend
+) is shared by all
+threads in a process, the user is responsible for ensuring that only
+one thread at a time calls this routine or 
+.I MPI_Buffer_attach
+\&.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+The Fortran binding for this routine is different.  Because Fortran
+does not have pointers, it is impossible to provide a way to use the
+output of this routine to exchange buffers.  In this case, only the
+size field is set.
+
+.SH NOTES FOR C
+Even though the 
+.I bufferptr
+argument is declared as 
+.I void *
+, it is
+really the address of a void pointer.  See the rationale in the
+standard for more details.
+
+.SH SEE ALSO
+MPI_Buffer_attach
+.br

macx64/mpi/mpich/share/man/man3/MPI_Cancel.3

@@ -0,0 +1,133 @@
+.TH MPI_Cancel 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Cancel \-  Cancels a communication request 
+.SH SYNOPSIS
+.nf
+int MPI_Cancel(MPI_Request * request)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B request 
+- communication request (handle)
+.PD 1
+
+.SH NOTES
+The primary expected use of 
+.I MPI_Cancel
+is in multi-buffering
+schemes, where speculative 
+.I MPI_Irecvs
+are made.  When the computation
+completes, some of these receive requests may remain; using 
+.I MPI_Cancel
+allows
+the user to cancel these unsatisfied requests.
+
+Cancelling a send operation is much more difficult, in large part because the
+send will usually be at least partially complete (the information on the tag,
+size, and source are usually sent immediately to the destination).
+Users are
+advised that cancelling a send, while a local operation (as defined by the MPI
+standard), is likely to be expensive (usually generating one or more internal
+messages).
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH NULL HANDLES
+The MPI 1.1 specification, in the section on opaque objects, explicitly
+disallows freeing a null communicator.  The text from the standard is:
+.nf
+A null handle argument is an erroneous IN argument in MPI calls, unless an
+exception is explicitly stated in the text that defines the function. Such
+exception is allowed for handles to request objects in Wait and Test calls
+(sections Communication Completion and Multiple Completions ). Otherwise, a
+null handle can only be passed to a function that allocates a new object and
+returns a reference to it in the handle.
+.fi
+
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_REQUEST 
+- Invalid 
+.I MPI_Request
+\&.
+Either null or, in the case of a
+.I MPI_Start
+or 
+.I MPI_Startall
+, not a persistent request.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Cart_coords.3

@@ -0,0 +1,141 @@
+.TH MPI_Cart_coords 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Cart_coords \-  Determines process coords in cartesian topology given rank in group 
+.SH SYNOPSIS
+.nf
+int MPI_Cart_coords(MPI_Comm comm, int rank, int maxdims, int coords[])
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator with cartesian structure (handle)
+.PD 1
+.PD 0
+.TP
+.B rank 
+- rank of a process within group of 
+.I comm
+(integer)
+.PD 1
+.PD 0
+.TP
+.B maxdims 
+- length of vector 
+.I coords
+in the calling program (integer)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B coords 
+- integer array (of size 
+.I ndims
+) containing the Cartesian
+coordinates of specified process (integer)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is both thread- and interrupt-safe.
+This means that this routine may safely be used by multiple threads and
+from within a signal handler.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TOPOLOGY 
+- Invalid topology.  Either there is no topology 
+associated with this communicator, or it is not the correct type (e.g.,
+.I MPI_CART
+when expecting 
+.I MPI_GRAPH
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_RANK 
+- Invalid source or destination rank.  Ranks must be between
+zero and the size of the communicator minus one; ranks in a receive
+(
+.I MPI_Recv
+, 
+.I MPI_Irecv
+, 
+.I MPI_Sendrecv
+, etc.) may also be 
+.I MPI_ANY_SOURCE
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_DIMS 
+- Invalid dimension argument.  A dimension argument
+is null or its length is less than or equal to zero.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Cart_create.3

@@ -0,0 +1,139 @@
+.TH MPI_Cart_create 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Cart_create \-  Makes a new communicator to which topology information has been attached 
+.SH SYNOPSIS
+.nf
+int MPI_Cart_create(MPI_Comm comm_old, int ndims, const int dims[],
+                    const int periods[], int reorder, MPI_Comm * comm_cart)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm_old 
+- input communicator (handle)
+.PD 1
+.PD 0
+.TP
+.B ndims 
+- number of dimensions of cartesian grid (integer)
+.PD 1
+.PD 0
+.TP
+.B dims 
+- integer array of size ndims specifying the number of processes in
+each dimension
+.PD 1
+.PD 0
+.TP
+.B periods 
+- logical array of size ndims specifying whether the grid is
+periodic (true) or not (false) in each dimension
+.PD 1
+.PD 0
+.TP
+.B reorder 
+- ranking may be reordered (true) or not (false) (logical)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B comm_cart 
+- communicator with new cartesian topology (handle)
+.PD 1
+
+.SH ALGORITHM
+We ignore 
+.I reorder
+info currently.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TOPOLOGY 
+- Invalid topology.  Either there is no topology 
+associated with this communicator, or it is not the correct type (e.g.,
+.I MPI_CART
+when expecting 
+.I MPI_GRAPH
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_DIMS 
+- Invalid dimension argument.  A dimension argument
+is null or its length is less than or equal to zero.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Cart_get.3

@@ -0,0 +1,136 @@
+.TH MPI_Cart_get 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Cart_get \-  Retrieves Cartesian topology information associated with a communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Cart_get(MPI_Comm comm, int maxdims, int dims[], int periods[], int coords[])
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator with cartesian structure (handle)
+.PD 1
+.PD 0
+.TP
+.B maxdims 
+- length of vectors  
+.I dims
+, 
+.I periods
+, and 
+.I coords
+in the calling program (integer)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B dims 
+- number of processes for each cartesian dimension (array of integer)
+.PD 1
+.PD 0
+.TP
+.B periods 
+- periodicity (true/false) for each cartesian dimension
+(array of logical)
+.PD 1
+.PD 0
+.TP
+.B coords 
+- coordinates of calling process in cartesian structure
+(array of integer)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TOPOLOGY 
+- Invalid topology.  Either there is no topology 
+associated with this communicator, or it is not the correct type (e.g.,
+.I MPI_CART
+when expecting 
+.I MPI_GRAPH
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Cart_map.3

@@ -0,0 +1,130 @@
+.TH MPI_Cart_map 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Cart_map \-  Maps process to Cartesian topology information 
+.SH SYNOPSIS
+.nf
+int MPI_Cart_map(MPI_Comm comm, int ndims, const int dims[], const int periods[], int *newrank)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- input communicator (handle)
+.PD 1
+.PD 0
+.TP
+.B ndims 
+- number of dimensions of Cartesian structure (integer)
+.PD 1
+.PD 0
+.TP
+.B dims 
+- integer array of size 
+.I ndims
+specifying the number of processes in
+each coordinate direction
+.PD 1
+.PD 0
+.TP
+.B periods 
+- logical array of size 
+.I ndims
+specifying the periodicity
+specification in each coordinate direction
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newrank 
+- reordered rank of the calling process; 
+.I MPI_UNDEFINED
+if
+calling process does not belong to grid (integer)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is both thread- and interrupt-safe.
+This means that this routine may safely be used by multiple threads and
+from within a signal handler.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_DIMS 
+- Invalid dimension argument.  A dimension argument
+is null or its length is less than or equal to zero.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Cart_rank.3

@@ -0,0 +1,136 @@
+.TH MPI_Cart_rank 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Cart_rank \-  Determines process rank in communicator given Cartesian location 
+.SH SYNOPSIS
+.nf
+int MPI_Cart_rank(MPI_Comm comm, const int coords[], int *rank)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator with cartesian structure (handle)
+.PD 1
+.PD 0
+.TP
+.B coords 
+- integer array (of size 
+.I ndims
+, the number of dimensions of
+the Cartesian topology associated with 
+.I comm
+) specifying the cartesian
+coordinates of a process
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B rank 
+- rank of specified process (integer)
+.PD 1
+
+.SH NOTES
+Out-of-range coordinates are erroneous for non-periodic dimensions.
+Versions of MPICH before 1.2.2 returned 
+.I MPI_PROC_NULL
+for the rank in this
+case.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is both thread- and interrupt-safe.
+This means that this routine may safely be used by multiple threads and
+from within a signal handler.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TOPOLOGY 
+- Invalid topology.  Either there is no topology 
+associated with this communicator, or it is not the correct type (e.g.,
+.I MPI_CART
+when expecting 
+.I MPI_GRAPH
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_RANK 
+- Invalid source or destination rank.  Ranks must be between
+zero and the size of the communicator minus one; ranks in a receive
+(
+.I MPI_Recv
+, 
+.I MPI_Irecv
+, 
+.I MPI_Sendrecv
+, etc.) may also be 
+.I MPI_ANY_SOURCE
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Cart_shift.3

@@ -0,0 +1,133 @@
+.TH MPI_Cart_shift 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Cart_shift \-  Returns the shifted source and destination ranks, given a shift direction and amount 
+.SH SYNOPSIS
+.nf
+int MPI_Cart_shift(MPI_Comm comm, int direction, int disp, int *rank_source, int *rank_dest)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator with cartesian structure (handle)
+.PD 1
+.PD 0
+.TP
+.B direction 
+- coordinate dimension of shift (integer)
+.PD 1
+.PD 0
+.TP
+.B disp 
+- displacement (> 0: upwards shift, < 0: downwards shift) (integer)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B rank_source 
+- rank of source process (integer)
+.PD 1
+.PD 0
+.TP
+.B rank_dest 
+- rank of destination process (integer)
+.PD 1
+
+.SH NOTES
+The 
+.I direction
+argument is in the range 
+.I [0,n-1]
+for an n-dimensional
+Cartesian mesh.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is both thread- and interrupt-safe.
+This means that this routine may safely be used by multiple threads and
+from within a signal handler.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TOPOLOGY 
+- Invalid topology.  Either there is no topology 
+associated with this communicator, or it is not the correct type (e.g.,
+.I MPI_CART
+when expecting 
+.I MPI_GRAPH
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Cart_sub.3

@@ -0,0 +1,125 @@
+.TH MPI_Cart_sub 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Cart_sub \-  Partitions a communicator into subgroups which form lower-dimensional cartesian subgrids 
+.SH SYNOPSIS
+.nf
+int MPI_Cart_sub(MPI_Comm comm, const int remain_dims[], MPI_Comm * newcomm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator with cartesian structure (handle)
+.PD 1
+.PD 0
+.TP
+.B remain_dims 
+- the  
+.I i
+th entry of remain_dims specifies whether the 
+.I i
+th
+dimension is kept in the subgrid (true) or is dropped (false) (logical
+vector)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- communicator containing the subgrid that includes the calling
+process (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TOPOLOGY 
+- Invalid topology.  Either there is no topology 
+associated with this communicator, or it is not the correct type (e.g.,
+.I MPI_CART
+when expecting 
+.I MPI_GRAPH
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Cartdim_get.3

@@ -0,0 +1,100 @@
+.TH MPI_Cartdim_get 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Cartdim_get \-  Retrieves Cartesian topology information associated with a communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Cartdim_get(MPI_Comm comm, int *ndims)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator with cartesian structure (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B ndims 
+- number of dimensions of the cartesian structure (integer)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is both thread- and interrupt-safe.
+This means that this routine may safely be used by multiple threads and
+from within a signal handler.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Close_port.3

@@ -0,0 +1,86 @@
+.TH MPI_Close_port 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Close_port \-  close port 
+.SH SYNOPSIS
+.nf
+int MPI_Close_port(const char *port_name)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B port_name 
+- a port name (string)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+The user is responsible for ensuring that multiple threads do not try to
+update the same MPI object from different threads.  This routine should
+not be used from within a signal handler.
+
+The MPI standard defined a thread-safe interface but this does not
+mean that all routines may be called without any thread locks.  For
+example, two threads must not attempt to change the contents of the
+same 
+.I MPI_Info
+object concurrently.  The user is responsible in this
+case for using some mechanism, such as thread locks, to ensure that
+only one thread at a time makes use of this routine.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_accept.3

@@ -0,0 +1,116 @@
+.TH MPI_Comm_accept 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_accept \-  Accept a request to form a new intercommunicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_accept(const char *port_name, MPI_Info info, int root, MPI_Comm comm,
+                    MPI_Comm * newcomm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B port_name 
+- port name (string, used only on root)
+.PD 1
+.PD 0
+.TP
+.B info 
+- implementation-dependent information (handle, used only on root)
+.PD 1
+.PD 0
+.TP
+.B root 
+- rank in comm of root node (integer)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- intracommunicator over which call is collective (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- intercommunicator with client as remote group (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_INFO 
+- Invalid Info 
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_call_errhandler.3

@@ -0,0 +1,105 @@
+.TH MPI_Comm_call_errhandler 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_call_errhandler \-  Call the error handler installed on a communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_call_errhandler(MPI_Comm comm, int errorcode)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator with error handler (handle)
+.PD 1
+.PD 0
+.TP
+.B errorcode 
+- error code (integer)
+.PD 1
+
+.SH NOTE
+Assuming the input parameters are valid, when the error handler is set to
+MPI_ERRORS_RETURN, this routine will always return MPI_SUCCESS.
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread and interrupt safe only if no MPI routine that
+updates or frees the same MPI object may be called concurrently
+with this routine.
+
+The MPI standard defined a thread-safe interface but this does not
+mean that all routines may be called without any thread locks.  For
+example, two threads must not attempt to change the contents of the
+same 
+.I MPI_Info
+object concurrently.  The user is responsible in this
+case for using some mechanism, such as thread locks, to ensure that
+only one thread at a time makes use of this routine.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_compare.3

@@ -0,0 +1,134 @@
+.TH MPI_Comm_compare 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_compare \-  Compares two communicators 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_compare(MPI_Comm comm1, MPI_Comm comm2, int *result)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm1 
+- comm1 (handle)
+.PD 1
+.PD 0
+.TP
+.B comm2 
+- comm2 (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B result 
+- integer which is 
+.I MPI_IDENT
+if the contexts and groups are the
+same, 
+.I MPI_CONGRUENT
+if different contexts but identical groups, 
+.I MPI_SIMILAR
+if different contexts but similar groups, and 
+.I MPI_UNEQUAL
+otherwise
+.PD 1
+
+.SH USING 'MPI_COMM_NULL' WITH 'MPI_COMM_COMPARE'
+
+It is an error to use 
+.I MPI_COMM_NULL
+as one of the arguments to
+.I MPI_Comm_compare
+\&.
+The relevant sections of the MPI standard are
+
+\&.
+(2.4.1 Opaque Objects)
+A null handle argument is an erroneous 
+.I IN
+argument in MPI calls, unless an
+exception is explicitly stated in the text that defines the function.
+
+\&.
+(5.4.1. Communicator Accessors)
+where there is no text in 
+.I MPI_COMM_COMPARE
+allowing a null handle.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+(To perform the communicator comparisions, this routine may need to
+allocate some memory.  Memory allocation is not interrupt-safe, and hence
+this routine is only thread-safe.)
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_connect.3

@@ -0,0 +1,121 @@
+.TH MPI_Comm_connect 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_connect \-  Make a request to form a new intercommunicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_connect(const char *port_name, MPI_Info info, int root, MPI_Comm comm,
+                     MPI_Comm * newcomm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B port_name 
+- network address (string, used only on root)
+.PD 1
+.PD 0
+.TP
+.B info 
+- implementation-dependent information (handle, used only on root)
+.PD 1
+.PD 0
+.TP
+.B root 
+- rank in comm of root node (integer)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- intracommunicator over which call is collective (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- intercommunicator with server as remote group (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_INFO 
+- Invalid Info 
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_PORT 
+- 
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_create.3

@@ -0,0 +1,111 @@
+.TH MPI_Comm_create 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_create \-  Creates a new communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_create(MPI_Comm comm, MPI_Group group, MPI_Comm * newcomm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+.PD 0
+.TP
+.B group 
+- group, which is a subset of the group of 
+.I comm
+(handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- new communicator (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_GROUP 
+- Null or invalid group passed to function.  
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_free
+.br

macx64/mpi/mpich/share/man/man3/MPI_Comm_create_errhandler.3

@@ -0,0 +1,109 @@
+.TH MPI_Comm_create_errhandler 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_create_errhandler \-  Create a communicator error handler 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_create_errhandler(MPI_Comm_errhandler_function * comm_errhandler_fn,
+                               MPI_Errhandler * errhandler)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm_errhandler_fn 
+- user defined error handling procedure (function)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B errhandler 
+- MPI error handler (handle)
+.PD 1
+
+.SH ERROR HANDLER
+The error handler function should be of the form
+
+void MPI_Comm_errhandler_function(MPI_Comm *comm, int *rc);
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_OTHER 
+- Other error; use 
+.I MPI_Error_string
+to get more information
+about this error code. 
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_create_group.3

@@ -0,0 +1,116 @@
+.TH MPI_Comm_create_group 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_create_group \-  Creates a new communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_create_group(MPI_Comm comm, MPI_Group group, int tag, MPI_Comm * newcomm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+.PD 0
+.TP
+.B group 
+- group, which is a subset of the group of 
+.I comm
+(handle)
+.PD 1
+.PD 0
+.TP
+.B tag 
+- safe tag unused by other communication
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- new communicator (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_GROUP 
+- Null or invalid group passed to function.  
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_free
+.br

macx64/mpi/mpich/share/man/man3/MPI_Comm_create_keyval.3

@@ -0,0 +1,147 @@
+.TH MPI_Comm_create_keyval 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_create_keyval \-  Create a new attribute key 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_create_keyval(MPI_Comm_copy_attr_function * comm_copy_attr_fn,
+                           MPI_Comm_delete_attr_function * comm_delete_attr_fn,
+                           int *comm_keyval, void *extra_state)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm_copy_attr_fn 
+- Copy callback function for 
+.I keyval
+
+.PD 1
+.PD 0
+.TP
+.B comm_delete_attr_fn 
+- Delete callback function for 
+.I keyval
+
+.PD 1
+.PD 0
+.TP
+.B extra_state 
+- Extra state for callback functions
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B comm_keyval 
+- key value for future access (integer)
+.PD 1
+
+.SH NOTES
+Key values are global (available for any and all communicators).
+
+Default copy and delete functions are available.  These are
+.PD 0
+.TP
+.B MPI_COMM_NULL_COPY_FN   
+- empty copy function
+.PD 1
+.PD 0
+.TP
+.B MPI_COMM_NULL_DELETE_FN 
+- empty delete function
+.PD 1
+.PD 0
+.TP
+.B MPI_COMM_DUP_FN         
+- simple dup function
+.PD 1
+
+There are subtle differences between C and Fortran that require that the
+copy_fn be written in the same language from which 
+.I MPI_Comm_create_keyval
+is called.
+This should not be a problem for most users; only programmers using both
+Fortran and C in the same program need to be sure that they follow this rule.
+
+
+.SH RETURN VALUE FROM ATTRIBUTE CALLBACKS
+The MPI-2 versions of the attribute callbacks should return either
+.I MPI_SUCCESS
+on success or a valid MPI error code or class on failure.
+The MPI standard is ambiguous on this point, but as MPI-2 provides
+the routines 
+.I MPI_Add_error_class
+and 
+.I MPI_Add_error_code
+that allow the
+user to define and use MPI error codes and classes.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_free_keyval
+.br

macx64/mpi/mpich/share/man/man3/MPI_Comm_delete_attr.3

@@ -0,0 +1,103 @@
+.TH MPI_Comm_delete_attr 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_delete_attr \-  Deletes an attribute value associated with a key on a  communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_delete_attr(MPI_Comm comm, int comm_keyval)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator to which attribute is attached (handle)
+.PD 1
+.PD 0
+.TP
+.B comm_keyval 
+- The key value of the deleted attribute (integer)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- This error class is associated with an error code that 
+indicates that an attempt was made to free one of the permanent keys.
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_set_attr, MPI_Comm_create_keyval
+.br

macx64/mpi/mpich/share/man/man3/MPI_Comm_disconnect.3

@@ -0,0 +1,99 @@
+.TH MPI_Comm_disconnect 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_disconnect \-  Disconnect from a communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_disconnect(MPI_Comm * comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH NOTES
+This routine waits for all pending communication to complete, then frees the
+communicator and sets 
+.I comm
+to 
+.I MPI_COMM_NULL
+\&.
+It may not be called
+with 
+.I MPI_COMM_WORLD
+or 
+.I MPI_COMM_SELF
+\&.
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_connect, MPI_Comm_join
+.br

macx64/mpi/mpich/share/man/man3/MPI_Comm_dup.3

@@ -0,0 +1,133 @@
+.TH MPI_Comm_dup 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_dup \-  Duplicates an existing communicator with all its cached information 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_dup(MPI_Comm comm, MPI_Comm * newcomm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- Communicator to be duplicated (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- A new communicator over the same group as 
+.I comm
+but with a new
+context. See notes.  (handle)
+.PD 1
+
+.SH NOTES
+This routine is used to create a new communicator that has a new
+communication context but contains the same group of processes as
+the input communicator.  Since all MPI communication is performed
+within a communicator (specifies as the group of processes 
+.B plus
+the context), this routine provides an effective way to create a
+private communicator for use by a software module or library.  In
+particular, no library routine should use 
+.I MPI_COMM_WORLD
+as the
+communicator; instead, a duplicate of a user-specified communicator
+should always be used.  For more information, see Using MPI, 2nd
+edition.
+
+Because this routine essentially produces a copy of a communicator,
+it also copies any attributes that have been defined on the input
+communicator, using the attribute copy function specified by the
+.I copy_function
+argument to 
+.I MPI_Keyval_create
+\&.
+This is
+particularly useful for (a) attributes that describe some property
+of the group associated with the communicator, such as its
+interconnection topology and (b) communicators that are given back
+to the user; the attibutes in this case can track subsequent
+.I MPI_Comm_dup
+operations on this communicator.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_free, MPI_Keyval_create, MPI_Attr_put, MPI_Attr_delete,
+.br
+MPI_Comm_create_keyval, MPI_Comm_set_attr, MPI_Comm_delete_attr

macx64/mpi/mpich/share/man/man3/MPI_Comm_dup_with_info.3

@@ -0,0 +1,115 @@
+.TH MPI_Comm_dup_with_info 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_dup_with_info \-  Duplicates an existing communicator with all its cached information 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_dup_with_info(MPI_Comm comm, MPI_Info info, MPI_Comm * newcomm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- Communicator to be duplicated (handle)
+.PD 1
+.PD 0
+.TP
+.B info 
+- info object (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- A new communicator over the same group as 
+.I comm
+but with a new
+context. See notes.  (handle)
+.PD 1
+
+.SH NOTES
+MPI_COMM_DUP_WITH_INFO behaves exactly as MPI_COMM_DUP except that
+the info hints associated with the communicator comm are not
+duplicated in newcomm.  The hints provided by the argument info are
+associated with the output communicator newcomm instead.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_dup, MPI_Comm_free, MPI_Keyval_create,
+.br
+MPI_Attr_put, MPI_Attr_delete, MPI_Comm_create_keyval,
+MPI_Comm_set_attr, MPI_Comm_delete_attr

macx64/mpi/mpich/share/man/man3/MPI_Comm_free.3

@@ -0,0 +1,118 @@
+.TH MPI_Comm_free 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_free \-  Marks the communicator object for deallocation 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_free(MPI_Comm * comm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- Communicator to be destroyed (handle)
+.PD 1
+
+.SH NOTES
+This routine 
+.B frees
+a communicator.  Because the communicator may still
+be in use by other MPI routines, the actual communicator storage will not
+be freed until all references to this communicator are removed.  For most
+users, the effect of this routine is the same as if it was in fact freed
+at this time of this call.
+
+.SH NULL HANDLES
+The MPI 1.1 specification, in the section on opaque objects, explicitly
+.SH DISALLOWS FREEING A NULL COMMUNICATOR.  THE TEXT FROM THE STANDARD IS
+.nf
+A null handle argument is an erroneous IN argument in MPI calls, unless an
+exception is explicitly stated in the text that defines the function. Such
+exception is allowed for handles to request objects in Wait and Test calls
+(sections Communication Completion and Multiple Completions). Otherwise, a
+null handle can only be passed to a function that allocates a new object and
+returns a reference to it in the handle.
+.fi
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_free_keyval.3

@@ -0,0 +1,97 @@
+.TH MPI_Comm_free_keyval 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_free_keyval \-  Frees an attribute key for communicators 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_free_keyval(int *comm_keyval)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm_keyval 
+- Frees the integer key value (integer)
+.PD 1
+
+.SH NOTES
+Key values are global (they can be used with any and all communicators)
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- This error class is associated with an error code that 
+indicates that an attempt was made to free one of the permanent keys.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_get_attr.3

@@ -0,0 +1,130 @@
+.TH MPI_Comm_get_attr 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_get_attr \-  Retrieves attribute value by key 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_get_attr(MPI_Comm comm, int comm_keyval, void *attribute_val, int *flag)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator to which attribute is attached (handle)
+.PD 1
+.PD 0
+.TP
+.B comm_keyval 
+- key value (integer)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B attribute_val 
+- attribute value, unless 
+.I flag
+= false
+.PD 1
+.PD 0
+.TP
+.B flag 
+- true if an attribute value was extracted;  false if no attribute is
+associated with the key
+.PD 1
+
+.SH NOTES
+Attributes must be extracted from the same language as they were inserted
+in with 
+.I MPI_Comm_set_attr
+\&.
+The notes for C and Fortran below explain
+why.
+
+.SH NOTES FOR C
+Even though the 
+.I attr_value
+argument is declared as 
+.I void *
+, it is
+really the address of a void pointer.  See the rationale in the
+standard for more details.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_KEYVAL 
+- Invalid keyval
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_get_errhandler.3

@@ -0,0 +1,103 @@
+.TH MPI_Comm_get_errhandler 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_get_errhandler \-  Get the error handler attached to a communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_get_errhandler(MPI_Comm comm, MPI_Errhandler * errhandler)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B errhandler 
+- handler currently associated with communicator (handle)
+.PD 1
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread and interrupt safe only if no MPI routine that
+updates or frees the same MPI object may be called concurrently
+with this routine.
+
+The MPI standard defined a thread-safe interface but this does not
+mean that all routines may be called without any thread locks.  For
+example, two threads must not attempt to change the contents of the
+same 
+.I MPI_Info
+object concurrently.  The user is responsible in this
+case for using some mechanism, such as thread locks, to ensure that
+only one thread at a time makes use of this routine.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_get_info.3

@@ -0,0 +1,107 @@
+.TH MPI_Comm_get_info 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_get_info \-  Returns a new info object containing the hints of the communicator associated with comm. The current setting of all hints actually used by the system related to this communicator is returned in info_used.  If no such hints exist, a handle to a newly created info object is returned that contains no key/value pair.  The user is responsible for freeing info_used via MPI_INFO_FREE. 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_get_info(MPI_Comm comm, MPI_Info * info_used)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator object (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B info_used 
+- new info argument (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_INFO 
+- Invalid Info 
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_OTHER 
+- Other error; use 
+.I MPI_Error_string
+to get more information
+about this error code. 
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_get_name.3

@@ -0,0 +1,121 @@
+.TH MPI_Comm_get_name 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_get_name \-  Return the print name from the communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_get_name(MPI_Comm comm, char *comm_name, int *resultlen)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- Communicator to get name of (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B comm_name 
+- On output, contains the name of the communicator.  It must
+be an array of size at least 
+.I MPI_MAX_OBJECT_NAME
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B resultlen 
+- Number of characters in name
+.PD 1
+
+.SH NOTES
+
+Because MPI specifies that null objects (e.g., 
+.I MPI_COMM_NULL
+) are invalid
+as input to MPI routines unless otherwise specified, using 
+.I MPI_COMM_NULL
+as input to this routine is an error.
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread and interrupt safe only if no MPI routine that
+updates or frees the same MPI object may be called concurrently
+with this routine.
+
+The MPI standard defined a thread-safe interface but this does not
+mean that all routines may be called without any thread locks.  For
+example, two threads must not attempt to change the contents of the
+same 
+.I MPI_Info
+object concurrently.  The user is responsible in this
+case for using some mechanism, such as thread locks, to ensure that
+only one thread at a time makes use of this routine.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_get_parent.3

@@ -0,0 +1,114 @@
+.TH MPI_Comm_get_parent 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_get_parent \-  Return the parent communicator for this process 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_get_parent(MPI_Comm * parent)
+.fi
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B parent 
+- the parent communicator (handle)
+.PD 1
+
+.SH NOTES
+
+If a process was started with 
+.I MPI_Comm_spawn
+or 
+.I MPI_Comm_spawn_multiple
+,
+.I MPI_Comm_get_parent
+returns the parent intercommunicator of the current
+process. This parent intercommunicator is created implicitly inside of
+.I MPI_Init
+and is the same intercommunicator returned by 
+.I MPI_Comm_spawn
+in the parents.
+
+If the process was not spawned, 
+.I MPI_Comm_get_parent
+returns
+.I MPI_COMM_NULL
+\&.
+
+
+After the parent communicator is freed or disconnected, 
+.I MPI_Comm_get_parent
+returns 
+.I MPI_COMM_NULL
+\&.
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is both thread- and interrupt-safe.
+This means that this routine may safely be used by multiple threads and
+from within a signal handler.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_group.3

@@ -0,0 +1,103 @@
+.TH MPI_Comm_group 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_group \-  Accesses the group associated with given communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_group(MPI_Comm comm, MPI_Group * group)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- Communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B group 
+- Group in communicator (handle)
+.PD 1
+
+.SH NOTES
+Because MPI specifies that null objects (e.g., 
+.I MPI_COMM_NULL
+) are invalid
+as input to MPI routines unless otherwise specified, using 
+.I MPI_COMM_NULL
+as input to this routine is an error.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_idup.3

@@ -0,0 +1,87 @@
+.TH MPI_Comm_idup 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_idup \-  nonblocking communicator duplication 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_idup(MPI_Comm comm, MPI_Comm * newcomm, MPI_Request * request)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- copy of comm (handle)
+.PD 1
+.PD 0
+.TP
+.B request 
+- communication request (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+

macx64/mpi/mpich/share/man/man3/MPI_Comm_join.3

@@ -0,0 +1,109 @@
+.TH MPI_Comm_join 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_join \-  Create a communicator by joining two processes connected by a socket. 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_join(int fd, MPI_Comm * intercomm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B fd 
+- socket file descriptor
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B intercomm 
+- new intercommunicator (handle)
+.PD 1
+
+.SH NOTES
+The socket must be quiescent before 
+.I MPI_COMM_JOIN
+is called and after
+.I MPI_COMM_JOIN
+returns. More specifically, on entry to 
+.I MPI_COMM_JOIN
+, a
+read on the socket will not read any data that was written to the socket
+before the remote process called 
+.I MPI_COMM_JOIN
+\&.
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_rank.3

@@ -0,0 +1,94 @@
+.TH MPI_Comm_rank 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_rank \-  Determines the rank of the calling process in the communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_rank(MPI_Comm comm, int *rank)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B rank 
+- rank of the calling process in the group of 
+.I comm
+(integer)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is both thread- and interrupt-safe.
+This means that this routine may safely be used by multiple threads and
+from within a signal handler.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_remote_group.3

@@ -0,0 +1,102 @@
+.TH MPI_Comm_remote_group 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_remote_group \-  Accesses the remote group associated with the given inter-communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_remote_group(MPI_Comm comm, MPI_Group * group)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- Communicator (must be an intercommunicator) (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B group 
+- remote group of communicator (handle)
+.PD 1
+
+.SH NOTES
+The user is responsible for freeing the group when it is no longer needed.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+
+.SH SEE ALSO
+MPI_Group_free
+.br

macx64/mpi/mpich/share/man/man3/MPI_Comm_remote_size.3

@@ -0,0 +1,102 @@
+.TH MPI_Comm_remote_size 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_remote_size \-  Determines the size of the remote group associated with an inter-communictor 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_remote_size(MPI_Comm comm, int *size)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B size 
+- number of processes in the remote group of 
+.I comm
+(integer)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is both thread- and interrupt-safe.
+This means that this routine may safely be used by multiple threads and
+from within a signal handler.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_set_attr.3

@@ -0,0 +1,142 @@
+.TH MPI_Comm_set_attr 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_set_attr \-  Stores attribute value associated with a key 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_set_attr(MPI_Comm comm, int comm_keyval, void *attribute_val)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator to which attribute will be attached (handle)
+.PD 1
+.PD 0
+.TP
+.B comm_keyval 
+- key value, as returned by  
+.I MPI_Comm_create_keyval
+(integer)
+.PD 1
+.PD 0
+.TP
+.B attribute_val 
+- attribute value
+.PD 1
+
+.SH NOTES
+Values of the permanent attributes 
+.I MPI_TAG_UB
+, 
+.I MPI_HOST
+, 
+.I MPI_IO
+,
+.I MPI_WTIME_IS_GLOBAL
+, 
+.I MPI_UNIVERSE_SIZE
+, 
+.I MPI_LASTUSEDCODE
+, and
+.I MPI_APPNUM
+may not be changed.
+
+The type of the attribute value depends on whether C, C++, or Fortran
+is being used.
+In C and C++, an attribute value is a pointer (
+.I void *
+); in Fortran, it is an
+address-sized integer.
+
+If an attribute is already present, the delete function (specified when the
+corresponding keyval was created) will be called.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_KEYVAL 
+- Invalid keyval
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- This error class is associated with an error code that 
+indicates that an attempt was made to free one of the permanent keys.
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_create_keyval, MPI_Comm_delete_attr
+.br

macx64/mpi/mpich/share/man/man3/MPI_Comm_set_errhandler.3

@@ -0,0 +1,113 @@
+.TH MPI_Comm_set_errhandler 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_set_errhandler \-  Set the error handler for a communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_set_errhandler(MPI_Comm comm, MPI_Errhandler errhandler)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+.PD 0
+.TP
+.B errhandler 
+- new error handler for communicator (handle)
+.PD 1
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread and interrupt safe only if no MPI routine that
+updates or frees the same MPI object may be called concurrently
+with this routine.
+
+The MPI standard defined a thread-safe interface but this does not
+mean that all routines may be called without any thread locks.  For
+example, two threads must not attempt to change the contents of the
+same 
+.I MPI_Info
+object concurrently.  The user is responsible in this
+case for using some mechanism, such as thread locks, to ensure that
+only one thread at a time makes use of this routine.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_OTHER 
+- Other error; use 
+.I MPI_Error_string
+to get more information
+about this error code. 
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_get_errhandler, MPI_Comm_call_errhandler
+.br

macx64/mpi/mpich/share/man/man3/MPI_Comm_set_info.3

@@ -0,0 +1,105 @@
+.TH MPI_Comm_set_info 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_set_info \-  Set new values for the hints of the communicator associated with comm.  The call is collective on the group of comm.  The info object may be different on each process, but any info entries that an implementation requires to be the same on all processes must appear with the same value in each process' info object. 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_set_info(MPI_Comm comm, MPI_Info info)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator object (handle)
+.PD 1
+.PD 0
+.TP
+.B info 
+- info argument (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_INFO 
+- Invalid Info 
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_OTHER 
+- Other error; use 
+.I MPI_Error_string
+to get more information
+about this error code. 
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_set_name.3

@@ -0,0 +1,101 @@
+.TH MPI_Comm_set_name 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_set_name \-  Sets the print name for a communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_set_name(MPI_Comm comm, const char *comm_name)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator to name (handle)
+.PD 1
+.PD 0
+.TP
+.B comm_name 
+- Name for communicator
+.PD 1
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread and interrupt safe only if no MPI routine that
+updates or frees the same MPI object may be called concurrently
+with this routine.
+
+The MPI standard defined a thread-safe interface but this does not
+mean that all routines may be called without any thread locks.  For
+example, two threads must not attempt to change the contents of the
+same 
+.I MPI_Info
+object concurrently.  The user is responsible in this
+case for using some mechanism, such as thread locks, to ensure that
+only one thread at a time makes use of this routine.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_size.3

@@ -0,0 +1,117 @@
+.TH MPI_Comm_size 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_size \-  Determines the size of the group associated with a communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_size(MPI_Comm comm, int *size)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B size 
+- number of processes in the group of 
+.I comm
+(integer)
+.PD 1
+
+.SH NOTES
+
+.SH NULL HANDLES
+The MPI 1.1 specification, in the section on opaque objects, explicitly
+disallows freeing a null communicator.  The text from the standard is:
+.nf
+A null handle argument is an erroneous IN argument in MPI calls, unless an
+exception is explicitly stated in the text that defines the function. Such
+exception is allowed for handles to request objects in Wait and Test calls
+(sections Communication Completion and Multiple Completions ). Otherwise, a
+null handle can only be passed to a function that allocates a new object and
+returns a reference to it in the handle.
+.fi
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is both thread- and interrupt-safe.
+This means that this routine may safely be used by multiple threads and
+from within a signal handler.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_spawn.3

@@ -0,0 +1,147 @@
+.TH MPI_Comm_spawn 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_spawn \-  Spawn up to maxprocs instances of a single MPI application 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_spawn(const char *command, char *argv[], int maxprocs, MPI_Info info,
+                   int root, MPI_Comm comm, MPI_Comm * intercomm, int array_of_errcodes[])
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B command 
+- name of program to be spawned (string, significant only at root)
+.PD 1
+.PD 0
+.TP
+.B argv 
+- arguments to command (array of strings, significant only at root)
+.PD 1
+.PD 0
+.TP
+.B maxprocs 
+- maximum number of processes to start (integer, significant only
+at root)
+.PD 1
+.PD 0
+.TP
+.B info 
+- a set of key-value pairs telling the runtime system where and how
+to start the processes (handle, significant only at root)
+.PD 1
+.PD 0
+.TP
+.B root 
+- rank of process in which previous arguments are examined (integer)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- intracommunicator containing group of spawning processes (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B intercomm 
+- intercommunicator between original group and the
+newly spawned group (handle)
+.PD 1
+.PD 0
+.TP
+.B array_of_errcodes 
+- one code per process (array of integer)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_INFO 
+- Invalid Info 
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_SPAWN 
+- 
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_spawn_multiple.3

@@ -0,0 +1,148 @@
+.TH MPI_Comm_spawn_multiple 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_spawn_multiple \-  short description 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_spawn_multiple(int count, char *array_of_commands[],
+                            char **array_of_argv[], const int array_of_maxprocs[],
+                            const MPI_Info array_of_info[], int root, MPI_Comm comm,
+                            MPI_Comm * intercomm, int array_of_errcodes[])
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B count 
+- number of commands (positive integer, significant to MPI only at
+root
+.PD 1
+.PD 0
+.TP
+.B array_of_commands 
+- programs to be executed (array of strings, significant
+only at root)
+.PD 1
+.PD 0
+.TP
+.B array_of_argv 
+- arguments for commands (array of array of strings,
+significant only at root)
+.PD 1
+.PD 0
+.TP
+.B array_of_maxprocs 
+- maximum number of processes to start for each command
+(array of integer, significant only at root)
+.PD 1
+.PD 0
+.TP
+.B array_of_info 
+- info objects telling the runtime system where and how to
+start processes (array of handles, significant only at root)
+.PD 1
+.PD 0
+.TP
+.B root 
+- rank of process in which previous arguments are examined (integer)
+.PD 1
+.PD 0
+.TP
+.B comm 
+- intracommunicator containing group of spawning processes (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B intercomm 
+- intercommunicator between original group and newly spawned group
+(handle)
+.PD 1
+.PD 0
+.TP
+.B array_of_errcodes 
+- one error code per process (array of integer)
+.PD 1
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_INFO 
+- Invalid Info 
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_SPAWN 
+- 
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Comm_split.3

@@ -0,0 +1,135 @@
+.TH MPI_Comm_split 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_split \-  Creates new communicators based on colors and keys 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_split(MPI_Comm comm, int color, int key, MPI_Comm * newcomm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+.PD 0
+.TP
+.B color 
+- control of subset assignment (nonnegative integer).  Processes
+with the same color are in the same new communicator
+.PD 1
+.PD 0
+.TP
+.B key 
+- control of rank assignment (integer)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- new communicator (handle)
+.PD 1
+
+.SH NOTES
+The 
+.I color
+must be non-negative or 
+.I MPI_UNDEFINED
+\&.
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ALGORITHM
+.nf
+1. Use MPI_Allgather to get the color and key from each process
+2. Count the number of processes with the same color; create a
+communicator with that many processes.  If this process has
+.I MPI_UNDEFINED
+as the color, create a process with a single member.
+3. Use key to order the ranks
+.fi
+
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_INTERN 
+- This error is returned when some part of the MPICH 
+implementation is unable to acquire memory.  
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_free
+.br

macx64/mpi/mpich/share/man/man3/MPI_Comm_split_type.3

@@ -0,0 +1,128 @@
+.TH MPI_Comm_split_type 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_split_type \-  Creates new communicators based on split types and keys 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_split_type(MPI_Comm comm, int split_type, int key, MPI_Info info, MPI_Comm * newcomm)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator (handle)
+.PD 1
+.PD 0
+.TP
+.B split_type 
+- type of processes to be grouped together (nonnegative integer).
+.PD 1
+.PD 0
+.TP
+.B key 
+- control of rank assignment (integer)
+.PD 1
+.PD 0
+.TP
+.B info 
+- hints to improve communicator creation (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B newcomm 
+- new communicator (handle)
+.PD 1
+
+.SH NOTES
+The 
+.I split_type
+must be non-negative or 
+.I MPI_UNDEFINED
+\&.
+
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_INTERN 
+- This error is returned when some part of the MPICH 
+implementation is unable to acquire memory.  
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_free
+.br

macx64/mpi/mpich/share/man/man3/MPI_Comm_test_inter.3

@@ -0,0 +1,100 @@
+.TH MPI_Comm_test_inter 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Comm_test_inter \-  Tests to see if a comm is an inter-communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Comm_test_inter(MPI_Comm comm, int *flag)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator to test (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B flag 
+- true if this is an inter-communicator(logical)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is both thread- and interrupt-safe.
+This means that this routine may safely be used by multiple threads and
+from within a signal handler.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Compare_and_swap.3

@@ -0,0 +1,177 @@
+.TH MPI_Compare_and_swap 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Compare_and_swap \-  Perform one-sided atomic compare-and-swap. 
+.SH SYNOPSIS
+.nf
+int MPI_Compare_and_swap(const void *origin_addr, const void *compare_addr,
+                         void *result_addr, MPI_Datatype datatype, int target_rank,
+                         MPI_Aint target_disp, MPI_Win win)
+.fi
+
+This function compares one element of type datatype in the compare buffer
+compare_addr with the buffer at offset target_disp in the target window
+specified by target_rank and win and replaces the value at the target with the
+value in the origin buffer origin_addr if the compare buffer and the target
+buffer are identical. The original value at the target is returned in the
+buffer result_addr.
+
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B origin_addr 
+- initial address of buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B compare_addr 
+- initial address of compare buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B result_addr 
+- initial address of result buffer (choice)
+.PD 1
+.PD 0
+.TP
+.B datatype 
+- datatype of the entry in origin, result, and target buffers (handle)
+.PD 1
+.PD 0
+.TP
+.B target_rank 
+- rank of target (nonnegative integer)
+.PD 1
+.PD 0
+.TP
+.B target_disp 
+- displacement from start of window to beginning of target buffer (non-negative integer)
+.PD 1
+.PD 0
+.TP
+.B win 
+- window object (handle)
+.PD 1
+
+.SH NOTES
+This operation is atomic with respect to other "accumulate" operations.
+
+The parameter datatype must belong to one of the following categories of
+predefined datatypes: C integer, Fortran integer, Logical, Multi-language
+types, or Byte as specified in Section 5.9.2 on page 176. The origin and result
+buffers (origin_addr and result_addr) must be disjoint.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COUNT 
+- Invalid count argument.  Count arguments must be 
+non-negative; a count of zero is often valid.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_OP 
+- Invalid operation.  MPI operations (objects of type 
+.I MPI_Op
+)
+must either be one of the predefined operations (e.g., 
+.I MPI_SUM
+) or
+created with 
+.I MPI_Op_create
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_RANK 
+- Invalid source or destination rank.  Ranks must be between
+zero and the size of the communicator minus one; ranks in a receive
+(
+.I MPI_Recv
+, 
+.I MPI_Irecv
+, 
+.I MPI_Sendrecv
+, etc.) may also be 
+.I MPI_ANY_SOURCE
+\&.
+
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_TYPE 
+- Invalid datatype argument.  Additionally, this error can
+occur if an uncommitted MPI_Datatype (see 
+.I MPI_Type_commit
+) is used
+in a communication call.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_WIN 
+- Invalid MPI window object
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Dims_create.3

@@ -0,0 +1,98 @@
+.TH MPI_Dims_create 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Dims_create \-  Creates a division of processors in a cartesian grid 
+.SH SYNOPSIS
+.nf
+int MPI_Dims_create(int nnodes, int ndims, int dims[])
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B nnodes 
+- number of nodes in a grid (integer)
+.PD 1
+.PD 0
+.TP
+.B ndims 
+- number of cartesian dimensions (integer)
+.PD 1
+
+.SH INPUT/OUTPUT PARAMETERS
+.PD 0
+.TP
+.B dims 
+- integer array of size  
+.I ndims
+specifying the number of nodes in each
+dimension.  A value of 0 indicates that 
+.I MPI_Dims_create
+should fill in a
+suitable value.
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Dist_graph_create.3

@@ -0,0 +1,146 @@
+.TH MPI_Dist_graph_create 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Dist_graph_create \-  MPI_DIST_GRAPH_CREATE returns a handle to a new communicator to which the distributed graph topology information is attached. 
+.SH SYNOPSIS
+.nf
+int MPI_Dist_graph_create(MPI_Comm comm_old, int n, const int sources[],
+                          const int degrees[], const int destinations[],
+                          const int weights[],
+                          MPI_Info info, int reorder, MPI_Comm * comm_dist_graph)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm_old 
+- input communicator (handle)
+.PD 1
+.PD 0
+.TP
+.B n 
+- number of source nodes for which this process specifies edges
+(non-negative integer)
+.PD 1
+.PD 0
+.TP
+.B sources 
+- array containing the n source nodes for which this process
+specifies edges (array of non-negative integers)
+.PD 1
+.PD 0
+.TP
+.B degrees 
+- array specifying the number of destinations for each source node
+in the source node array (array of non-negative integers)
+.PD 1
+.PD 0
+.TP
+.B destinations 
+- destination nodes for the source nodes in the source node
+array (array of non-negative integers)
+.PD 1
+.PD 0
+.TP
+.B weights 
+- weights for source to destination edges (array of non-negative
+integers or MPI_UNWEIGHTED)
+.PD 1
+.PD 0
+.TP
+.B info 
+- hints on optimization and interpretation of weights (handle)
+.PD 1
+.PD 0
+.TP
+.B reorder 
+- the process may be reordered (true) or not (false) (logical)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B comm_dist_graph 
+- communicator with distributed graph topology added (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_OTHER 
+- Other error; use 
+.I MPI_Error_string
+to get more information
+about this error code. 
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Dist_graph_create_adjacent.3

@@ -0,0 +1,152 @@
+.TH MPI_Dist_graph_create_adjacent 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Dist_graph_create_adjacent \-  returns a handle to a new communicator to which the distributed graph topology information is attached. 
+.SH SYNOPSIS
+.nf
+int MPI_Dist_graph_create_adjacent(MPI_Comm comm_old,
+                                   int indegree, const int sources[],
+                                   const int sourceweights[],
+                                   int outdegree, const int destinations[],
+                                   const int destweights[],
+                                   MPI_Info info, int reorder, MPI_Comm * comm_dist_graph)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm_old 
+- input communicator (handle)
+.PD 1
+.PD 0
+.TP
+.B indegree 
+- size of sources and sourceweights arrays (non-negative integer)
+.PD 1
+.PD 0
+.TP
+.B sources 
+- ranks of processes for which the calling process is a
+destination (array of non-negative integers)
+.PD 1
+.PD 0
+.TP
+.B sourceweights 
+- weights of the edges into the calling
+process (array of non-negative integers or MPI_UNWEIGHTED)
+.PD 1
+.PD 0
+.TP
+.B outdegree 
+- size of destinations and destweights arrays (non-negative integer)
+.PD 1
+.PD 0
+.TP
+.B destinations 
+- ranks of processes for which the calling process is a
+source (array of non-negative integers)
+.PD 1
+.PD 0
+.TP
+.B destweights 
+- weights of the edges out of the calling process
+(array of non-negative integers or MPI_UNWEIGHTED)
+.PD 1
+.PD 0
+.TP
+.B info 
+- hints on optimization and interpretation of weights (handle)
+.PD 1
+.PD 0
+.TP
+.B reorder 
+- the ranks may be reordered (true) or not (false) (logical)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B comm_dist_graph 
+- communicator with distributed graph topology (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_OTHER 
+- Other error; use 
+.I MPI_Error_string
+to get more information
+about this error code. 
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Dist_graph_neighbors.3

@@ -0,0 +1,114 @@
+.TH MPI_Dist_graph_neighbors 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Dist_graph_neighbors \-  Provides adjacency information for a distributed graph topology. 
+.SH SYNOPSIS
+.nf
+int MPI_Dist_graph_neighbors(MPI_Comm comm,
+                             int maxindegree, int sources[], int sourceweights[],
+                             int maxoutdegree, int destinations[], int destweights[])
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator with distributed graph topology (handle)
+.PD 1
+.PD 0
+.TP
+.B maxindegree 
+- size of sources and sourceweights arrays (non-negative integer)
+.PD 1
+.PD 0
+.TP
+.B maxoutdegree 
+- size of destinations and destweights arrays (non-negative integer)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B sources 
+- processes for which the calling process is a destination (array of non-negative integers)
+.PD 1
+.PD 0
+.TP
+.B sourceweights 
+- weights of the edges into the calling process (array of non-negative integers)
+.PD 1
+.PD 0
+.TP
+.B destinations 
+- processes for which the calling process is a source (array of non-negative integers)
+.PD 1
+.PD 0
+.TP
+.B destweights 
+- weights of the edges out of the calling process (array of non-negative integers)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Dist_graph_neighbors_count.3

@@ -0,0 +1,97 @@
+.TH MPI_Dist_graph_neighbors_count 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Dist_graph_neighbors_count \-  Provides adjacency information for a distributed graph topology. 
+.SH SYNOPSIS
+.nf
+int MPI_Dist_graph_neighbors_count(MPI_Comm comm, int *indegree, int *outdegree, int *weighted)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator with distributed graph topology (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B indegree 
+- number of edges into this process (non-negative integer)
+.PD 1
+.PD 0
+.TP
+.B outdegree 
+- number of edges out of this process (non-negative integer)
+.PD 1
+.PD 0
+.TP
+.B weighted 
+- false if MPI_UNWEIGHTED was supplied during creation, true otherwise (logical)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Errhandler_create.3

@@ -0,0 +1,119 @@
+.TH MPI_Errhandler_create 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Errhandler_create \-  Creates an MPI-style errorhandler 
+.SH SYNOPSIS
+.nf
+int MPI_Errhandler_create(MPI_Handler_function * function, MPI_Errhandler * errhandler)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B function 
+- user defined error handling procedure
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B errhandler 
+- MPI error handler (handle)
+.PD 1
+
+.SH NOTES
+The MPI Standard states that an implementation may make the output value
+(errhandler) simply the address of the function.  However, the action of
+.I MPI_Errhandler_free
+makes this impossible, since it is required to set the
+value of the argument to 
+.I MPI_ERRHANDLER_NULL
+\&.
+In addition, the actual
+error handler must remain until all communicators that use it are
+freed.
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH DEPRECATED FUNCTION
+The MPI-2 standard deprecated a number of routines because MPI-2 provides
+better versions.  This routine is one of those that was deprecated.  The
+routine may continue to be used, but new code should use the replacement
+routine.
+The replacement routine for this function is 
+.I MPI_Comm_create_errhandler
+\&.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_INTERN 
+- This error is returned when some part of the MPICH 
+implementation is unable to acquire memory.  
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_create_errhandler, MPI_Errhandler_free
+.br

macx64/mpi/mpich/share/man/man3/MPI_Errhandler_free.3

@@ -0,0 +1,91 @@
+.TH MPI_Errhandler_free 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Errhandler_free \-  Frees an MPI-style errorhandler 
+.SH SYNOPSIS
+.nf
+int MPI_Errhandler_free(MPI_Errhandler * errhandler)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B errhandler 
+- MPI error handler (handle).  Set to 
+.I MPI_ERRHANDLER_NULL
+on
+exit.
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Errhandler_get.3

@@ -0,0 +1,123 @@
+.TH MPI_Errhandler_get 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Errhandler_get \-  Gets the error handler for a communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Errhandler_get(MPI_Comm comm, MPI_Errhandler * errhandler)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator to get the error handler from (handle)
+.PD 1
+
+.SH OUTPUT PARAMETERS
+.PD 0
+.TP
+.B errhandler 
+- MPI error handler currently associated with communicator
+(handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH NOTE ON IMPLEMENTATION
+
+The MPI Standard was unclear on whether this routine required the user to call
+.I MPI_Errhandler_free
+once for each call made to this routine in order to
+free the error handler.  After some debate, the MPI Forum added an explicit
+statement that users are required to call 
+.I MPI_Errhandler_free
+when the
+return value from this routine is no longer needed.  This behavior is similar
+to the other MPI routines for getting objects; for example, 
+.I MPI_Comm_group
+requires that the user call 
+.I MPI_Group_free
+when the group returned
+by 
+.I MPI_Comm_group
+is no longer needed.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1

macx64/mpi/mpich/share/man/man3/MPI_Errhandler_set.3

@@ -0,0 +1,115 @@
+.TH MPI_Errhandler_set 3 "6/5/2019" " " "MPI"
+.SH NAME
+MPI_Errhandler_set \-  Sets the error handler for a communicator 
+.SH SYNOPSIS
+.nf
+int MPI_Errhandler_set(MPI_Comm comm, MPI_Errhandler errhandler)
+.fi
+.SH INPUT PARAMETERS
+.PD 0
+.TP
+.B comm 
+- communicator to set the error handler for (handle)
+.PD 1
+.PD 0
+.TP
+.B errhandler 
+- new MPI error handler for communicator (handle)
+.PD 1
+
+.SH THREAD AND INTERRUPT SAFETY
+
+This routine is thread-safe.  This means that this routine may be
+safely used by multiple threads without the need for any user-provided
+thread locks.  However, the routine is not interrupt safe.  Typically,
+this is due to the use of memory allocation routines such as 
+.I malloc
+or other non-MPICH runtime routines that are themselves not interrupt-safe.
+
+.SH DEPRECATED FUNCTION
+The MPI-2 standard deprecated a number of routines because MPI-2 provides
+better versions.  This routine is one of those that was deprecated.  The
+routine may continue to be used, but new code should use the replacement
+routine.
+The replacement for this routine is 
+.I MPI_Comm_set_errhandler
+\&.
+
+
+.SH NOTES FOR FORTRAN
+All MPI routines in Fortran (except for 
+.I MPI_WTIME
+and 
+.I MPI_WTICK
+) have
+an additional argument 
+.I ierr
+at the end of the argument list.  
+.I ierr
+is an integer and has the same meaning as the return value of the routine
+in C.  In Fortran, MPI routines are subroutines, and are invoked with the
+.I call
+statement.
+
+All MPI objects (e.g., 
+.I MPI_Datatype
+, 
+.I MPI_Comm
+) are of type 
+.I INTEGER
+in Fortran.
+
+.SH ERRORS
+
+All MPI routines (except 
+.I MPI_Wtime
+and 
+.I MPI_Wtick
+) return an error value;
+C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with 
+.I MPI_Comm_set_errhandler
+(for communicators),
+.I MPI_File_set_errhandler
+(for files), and 
+.I MPI_Win_set_errhandler
+(for
+RMA windows).  The MPI-1 routine 
+.I MPI_Errhandler_set
+may be used but
+its use is deprecated.  The predefined error handler
+.I MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does 
+.B not
+guarentee that an MPI program can continue past
+an error; however, MPI implementations will attempt to continue whenever
+possible.
+
+.PD 0
+.TP
+.B MPI_SUCCESS 
+- No error; MPI routine completed successfully.
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_COMM 
+- Invalid communicator.  A common error is to use a null
+communicator in a call (not even allowed in 
+.I MPI_Comm_rank
+).
+.PD 1
+.PD 0
+.TP
+.B MPI_ERR_ARG 
+- Invalid argument.  Some argument is invalid and is not
+identified by a specific error class (e.g., 
+.I MPI_ERR_RANK
+).
+.PD 1
+
+.SH SEE ALSO
+MPI_Comm_set_errhandler, MPI_Errhandler_create, MPI_Comm_create_errhandler
+.br