mac mpi

macx64/mpi/openmpi/share/man/man1/mpic++.1

@@ -0,0 +1,265 @@
+.\" Copyright (c) 2008      Sun Microsystems, Inc.  All rights reserved.
+.\" Copyright (c) 2011-2012 Cisco Systems, Inc.  All rights reserved.
+.TH mpic++ 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.
+.SH NAME
+mpic++ -- Open MPI C++ wrapper compiler
+.
+.SH SYNTAX
+mpic++ [-showme|-showme:compile|-showme:link] ...
+.
+.SH OPTIONS
+.TP
+--showme
+This option comes in several different variants (see below).  None of
+the variants invokes the underlying compiler; they all provide
+information on how the underlying compiler would have been invoked had
+.I --showme
+not been used.
+The basic
+.I --showme
+option outputs the command line that would be executed to compile the
+program.  \fBNOTE:\fR If a non-filename argument is passed on the
+command line, the \fI-showme\fR option will \fInot\fR display any
+additional flags.  For example, both "mpic++ --showme" and
+"mpic++ --showme my_source.c" will show all the wrapper-supplied
+flags.  But "mpic++ --showme -v" will only show the underlying
+compiler name and "-v".
+.TP
+--showme:compile
+Output the compiler flags that would have been supplied to the
+C++ compiler.
+.TP
+--showme:link
+Output the linker flags that would have been supplied to the
+C++ compiler.
+.TP
+--showme:command
+Outputs the underlying C++ compiler command (which may be one
+or more tokens).
+.TP
+--showme:incdirs
+Outputs a space-delimited (but otherwise undecorated) list of
+directories that the wrapper compiler would have provided to the
+underlying C++ compiler to indicate where relevant header files
+are located.
+.TP
+--showme:libdirs
+Outputs a space-delimited (but otherwise undecorated) list of
+directories that the wrapper compiler would have provided to the
+underlying linker to indicate where relevant libraries are located.
+.TP
+--showme:libs
+Outputs a space-delimited (but otherwise undecorated) list of library
+names that the wrapper compiler would have used to link an
+application.  For example: "mpi open-rte open-pal util".
+.TP
+--showme:version
+Outputs the version number of Open MPI.
+.TP
+--showme:help
+Output a brief usage help message.
+.PP
+See the man page for your underlying C++ compiler for other
+options that can be passed through mpic++.
+.
+.
+.SH DESCRIPTION
+.PP
+Conceptually, the role of these commands is quite simple:
+transparently add relevant compiler and linker flags to the user's
+command line that are necessary to compile / link Open MPI
+programs, and then invoke the underlying compiler to actually perform
+the command.
+.
+.PP
+As such, these commands are frequently referred to as "wrapper"
+compilers because they do not actually compile or link applications
+themselves; they only add in command line flags and invoke the
+back-end compiler.
+.
+.
+.SS Background
+Open MPI is comprised of three software layers: OPAL (Open Portable
+Access Layer), ORTE (Open Run-Time Environment), and OMPI (Open MPI).
+There are wrapper compilers for each layer; each layer's wrapper only
+links in the libraries relevant for that layer.  Specifically, each
+layer provides the following wrapper compilers:
+.
+.TP 4
+OPAL
+\fIopalcc\fR and \fIopalc++\fR
+.
+.TP
+ORTE
+\fIortecc\fR and \fIortec++\fR
+.
+.TP
+OMPI
+\fImpicc\fR, \fImpic++\fR, \fImpicxx\fR, \fImpiCC\fR (only on systems with
+case-senstive file systems), and \fImpifort\fR (and its legacy/deprecated
+names \fImpif77\fR and \fImpif90\fR).  Note
+that \fImpic++\fR, \fImpicxx\fR, and \fImpiCC\fR all invoke the same
+underlying C++ compiler with the same options.  All are provided as
+compatibility with other MPI implementations.
+.
+.
+.SS Fortran Notes
+.PP
+The Fortran wrapper compiler for MPI (\fImpifort\fR, and its
+legacy/deprecated names \fImpif77\fR and \fImpif90\fR) can compile and
+link MPI applications that use any/all of the MPI Fortran bindings:
+.IR mpif.h ,
+the
+.I mpi
+module, and the
+.I mpi_f08
+module (assuming Open MPI was installed with support for each of these
+Fortran bindings).  Specifically: it is no longer necessary to use
+different wrapper compilers for applications that use
+.I mpif.h
+vs. applications that use the
+.I mpi
+module -- just use
+.I mpifort
+for all Fortran MPI applications.
+.
+.PP
+Note, however, that the Fortran compiler may require additional
+command-line options to enforce a specific Fortran dialect.  For
+example, in some versions of the IBM XLF compiler, if xlf90 is the
+underlying Fortran compiler,
+.IR -qfixed
+may be necessary to compile fixed-format Fortran source files.
+.
+.PP
+Finally, note that
+.I mpifort
+will be inoperative and will return an error on use if Fortran support
+was not built into the MP Ilayer.
+.
+.
+.SS Overview
+\fImpic++\fR is a convenience wrappers for the underlying
+C++ compiler.  Translation of an Open MPI program requires the
+linkage of the Open MPI-specific libraries which may not reside in
+one of the standard search directories of ld(1).  It also often
+requires the inclusion of header files what may also not be found in a
+standard location.
+.
+.PP
+\fImpic++\fR passes its arguments to the underlying C++
+compiler along with the -I, -L and -l options required by Open MPI
+programs.
+.
+.PP
+The Open MPI Team \fIstrongly\fR encourages using the wrapper
+compilers instead of attempting to link to the Open MPI libraries
+manually.  This allows the specific implementation of Open MPI to
+change without forcing changes to linker directives in users'
+Makefiles.  Indeed, the specific set of flags and libraries used by
+the wrapper compilers depends on how Open MPI was configured and
+built; the values can change between different installations of the
+same version of Open MPI.
+.
+.PP
+Indeed, since the wrappers are simply thin shells on top of an
+underlying compiler, there are very, very few compelling reasons
+\fInot\fR to use \fImpic++\fR.  When it is not possible to use the
+wrappers directly, the \fI-showme:compile\fR and \fI-showme:link\fR
+options should be used to determine what flags the wrappers would have
+used.  For example:
+.
+.PP
+shell$ cc -c file1.c `mpicc -showme:compile`
+.
+.PP
+shell$ cc -c file2.c `mpicc -showme:compile`
+.
+.PP
+shell$ cc file1.o file2.o `mpicc -showme:link` -o my_mpi_program
+.
+.
+.SH NOTES
+.PP
+It is possible to make the wrapper compilers multi-lib aware.  That
+is, the libraries and includes specified may differ based on the
+compiler flags specified (for example, with the GNU compilers on
+Linux, a different library path may be used if -m32 is seen versus
+-m64 being seen).  This is not the default behavior in a standard
+build, but can be activated (for example, in a binary package
+providing both 32 and 64 bit support).  More information can be found
+at:
+.PP
+  https://github.com/open-mpi/ompi/wiki/compilerwrapper3264
+.
+.
+.SH FILES
+.PP
+The string that the wrapper compilers insert into the command line
+before invoking the underlying compiler are stored in a text file
+created by Open MPI and installed to
+\fI$pkgdata/mpic++-wrapper-data.txt\fR, where \fI$pkgdata\fR
+is typically \fI$prefix/share/openmpi\fR, and \fI$prefix\fR is the top
+installation directory of Open MPI.
+.
+.PP
+It is rarely necessary to edit this file, but it can be examined to
+gain insight into what flags the wrappers are placing on the command
+line.
+.
+.
+.SH ENVIRONMENT VARIABLES
+.PP
+By default, the wrappers use the compilers that were selected when
+Open MPI was configured.  These compilers were either found
+automatically by Open MPI's "configure" script, or were selected by
+the user in the CC, CXX, F77, and/or FC environment variables
+before "configure" was invoked.  Additionally, other arguments
+specific to the compiler may have been selected by configure.
+.
+.PP
+These values can be selectively overridden by either editing the text
+files containing this configuration information (see the \fBFILES\fR
+section), or by setting selected environment variables of the
+form "OMPI_value".
+.
+.PP
+Valid value names are:
+.
+.TP
+CPPFLAGS
+Flags added when invoking the preprocessor (C or C++)
+.
+.TP
+LDFLAGS
+Flags added when invoking the linker (C, C++, or Fortran)
+.
+.TP
+LIBS
+Libraries added when invoking the linker (C, C++, or Fortran)
+.
+.TP
+CC
+C compiler
+.
+.TP
+CFLAGS
+C compiler flags
+.
+.TP
+CXX
+C++ compiler
+.
+.TP
+CXXFLAGS
+C++ compiler flags
+.
+.
+.TP
+FC
+Fortran compiler
+.
+.TP
+FCFLAGS
+Fortran compiler flags

macx64/mpi/openmpi/share/man/man1/mpicc.1

@@ -0,0 +1,265 @@
+.\" Copyright (c) 2008      Sun Microsystems, Inc.  All rights reserved.
+.\" Copyright (c) 2011-2012 Cisco Systems, Inc.  All rights reserved.
+.TH mpicc 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.
+.SH NAME
+mpicc -- Open MPI C wrapper compiler
+.
+.SH SYNTAX
+mpicc [-showme|-showme:compile|-showme:link] ...
+.
+.SH OPTIONS
+.TP
+--showme
+This option comes in several different variants (see below).  None of
+the variants invokes the underlying compiler; they all provide
+information on how the underlying compiler would have been invoked had
+.I --showme
+not been used.
+The basic
+.I --showme
+option outputs the command line that would be executed to compile the
+program.  \fBNOTE:\fR If a non-filename argument is passed on the
+command line, the \fI-showme\fR option will \fInot\fR display any
+additional flags.  For example, both "mpicc --showme" and
+"mpicc --showme my_source.c" will show all the wrapper-supplied
+flags.  But "mpicc --showme -v" will only show the underlying
+compiler name and "-v".
+.TP
+--showme:compile
+Output the compiler flags that would have been supplied to the
+C compiler.
+.TP
+--showme:link
+Output the linker flags that would have been supplied to the
+C compiler.
+.TP
+--showme:command
+Outputs the underlying C compiler command (which may be one
+or more tokens).
+.TP
+--showme:incdirs
+Outputs a space-delimited (but otherwise undecorated) list of
+directories that the wrapper compiler would have provided to the
+underlying C compiler to indicate where relevant header files
+are located.
+.TP
+--showme:libdirs
+Outputs a space-delimited (but otherwise undecorated) list of
+directories that the wrapper compiler would have provided to the
+underlying linker to indicate where relevant libraries are located.
+.TP
+--showme:libs
+Outputs a space-delimited (but otherwise undecorated) list of library
+names that the wrapper compiler would have used to link an
+application.  For example: "mpi open-rte open-pal util".
+.TP
+--showme:version
+Outputs the version number of Open MPI.
+.TP
+--showme:help
+Output a brief usage help message.
+.PP
+See the man page for your underlying C compiler for other
+options that can be passed through mpicc.
+.
+.
+.SH DESCRIPTION
+.PP
+Conceptually, the role of these commands is quite simple:
+transparently add relevant compiler and linker flags to the user's
+command line that are necessary to compile / link Open MPI
+programs, and then invoke the underlying compiler to actually perform
+the command.
+.
+.PP
+As such, these commands are frequently referred to as "wrapper"
+compilers because they do not actually compile or link applications
+themselves; they only add in command line flags and invoke the
+back-end compiler.
+.
+.
+.SS Background
+Open MPI is comprised of three software layers: OPAL (Open Portable
+Access Layer), ORTE (Open Run-Time Environment), and OMPI (Open MPI).
+There are wrapper compilers for each layer; each layer's wrapper only
+links in the libraries relevant for that layer.  Specifically, each
+layer provides the following wrapper compilers:
+.
+.TP 4
+OPAL
+\fIopalcc\fR and \fIopalc++\fR
+.
+.TP
+ORTE
+\fIortecc\fR and \fIortec++\fR
+.
+.TP
+OMPI
+\fImpicc\fR, \fImpic++\fR, \fImpicxx\fR, \fImpiCC\fR (only on systems with
+case-senstive file systems), and \fImpifort\fR (and its legacy/deprecated
+names \fImpif77\fR and \fImpif90\fR).  Note
+that \fImpic++\fR, \fImpicxx\fR, and \fImpiCC\fR all invoke the same
+underlying C++ compiler with the same options.  All are provided as
+compatibility with other MPI implementations.
+.
+.
+.SS Fortran Notes
+.PP
+The Fortran wrapper compiler for MPI (\fImpifort\fR, and its
+legacy/deprecated names \fImpif77\fR and \fImpif90\fR) can compile and
+link MPI applications that use any/all of the MPI Fortran bindings:
+.IR mpif.h ,
+the
+.I mpi
+module, and the
+.I mpi_f08
+module (assuming Open MPI was installed with support for each of these
+Fortran bindings).  Specifically: it is no longer necessary to use
+different wrapper compilers for applications that use
+.I mpif.h
+vs. applications that use the
+.I mpi
+module -- just use
+.I mpifort
+for all Fortran MPI applications.
+.
+.PP
+Note, however, that the Fortran compiler may require additional
+command-line options to enforce a specific Fortran dialect.  For
+example, in some versions of the IBM XLF compiler, if xlf90 is the
+underlying Fortran compiler,
+.IR -qfixed
+may be necessary to compile fixed-format Fortran source files.
+.
+.PP
+Finally, note that
+.I mpifort
+will be inoperative and will return an error on use if Fortran support
+was not built into the MP Ilayer.
+.
+.
+.SS Overview
+\fImpicc\fR is a convenience wrappers for the underlying
+C compiler.  Translation of an Open MPI program requires the
+linkage of the Open MPI-specific libraries which may not reside in
+one of the standard search directories of ld(1).  It also often
+requires the inclusion of header files what may also not be found in a
+standard location.
+.
+.PP
+\fImpicc\fR passes its arguments to the underlying C
+compiler along with the -I, -L and -l options required by Open MPI
+programs.
+.
+.PP
+The Open MPI Team \fIstrongly\fR encourages using the wrapper
+compilers instead of attempting to link to the Open MPI libraries
+manually.  This allows the specific implementation of Open MPI to
+change without forcing changes to linker directives in users'
+Makefiles.  Indeed, the specific set of flags and libraries used by
+the wrapper compilers depends on how Open MPI was configured and
+built; the values can change between different installations of the
+same version of Open MPI.
+.
+.PP
+Indeed, since the wrappers are simply thin shells on top of an
+underlying compiler, there are very, very few compelling reasons
+\fInot\fR to use \fImpicc\fR.  When it is not possible to use the
+wrappers directly, the \fI-showme:compile\fR and \fI-showme:link\fR
+options should be used to determine what flags the wrappers would have
+used.  For example:
+.
+.PP
+shell$ cc -c file1.c `mpicc -showme:compile`
+.
+.PP
+shell$ cc -c file2.c `mpicc -showme:compile`
+.
+.PP
+shell$ cc file1.o file2.o `mpicc -showme:link` -o my_mpi_program
+.
+.
+.SH NOTES
+.PP
+It is possible to make the wrapper compilers multi-lib aware.  That
+is, the libraries and includes specified may differ based on the
+compiler flags specified (for example, with the GNU compilers on
+Linux, a different library path may be used if -m32 is seen versus
+-m64 being seen).  This is not the default behavior in a standard
+build, but can be activated (for example, in a binary package
+providing both 32 and 64 bit support).  More information can be found
+at:
+.PP
+  https://github.com/open-mpi/ompi/wiki/compilerwrapper3264
+.
+.
+.SH FILES
+.PP
+The string that the wrapper compilers insert into the command line
+before invoking the underlying compiler are stored in a text file
+created by Open MPI and installed to
+\fI$pkgdata/mpicc-wrapper-data.txt\fR, where \fI$pkgdata\fR
+is typically \fI$prefix/share/openmpi\fR, and \fI$prefix\fR is the top
+installation directory of Open MPI.
+.
+.PP
+It is rarely necessary to edit this file, but it can be examined to
+gain insight into what flags the wrappers are placing on the command
+line.
+.
+.
+.SH ENVIRONMENT VARIABLES
+.PP
+By default, the wrappers use the compilers that were selected when
+Open MPI was configured.  These compilers were either found
+automatically by Open MPI's "configure" script, or were selected by
+the user in the CC, CXX, F77, and/or FC environment variables
+before "configure" was invoked.  Additionally, other arguments
+specific to the compiler may have been selected by configure.
+.
+.PP
+These values can be selectively overridden by either editing the text
+files containing this configuration information (see the \fBFILES\fR
+section), or by setting selected environment variables of the
+form "OMPI_value".
+.
+.PP
+Valid value names are:
+.
+.TP
+CPPFLAGS
+Flags added when invoking the preprocessor (C or C++)
+.
+.TP
+LDFLAGS
+Flags added when invoking the linker (C, C++, or Fortran)
+.
+.TP
+LIBS
+Libraries added when invoking the linker (C, C++, or Fortran)
+.
+.TP
+CC
+C compiler
+.
+.TP
+CFLAGS
+C compiler flags
+.
+.TP
+CXX
+C++ compiler
+.
+.TP
+CXXFLAGS
+C++ compiler flags
+.
+.
+.TP
+FC
+Fortran compiler
+.
+.TP
+FCFLAGS
+Fortran compiler flags

macx64/mpi/openmpi/share/man/man1/mpicxx.1

@@ -0,0 +1,265 @@
+.\" Copyright (c) 2008      Sun Microsystems, Inc.  All rights reserved.
+.\" Copyright (c) 2011-2012 Cisco Systems, Inc.  All rights reserved.
+.TH mpicxx 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.
+.SH NAME
+mpicxx -- Open MPI C++ wrapper compiler
+.
+.SH SYNTAX
+mpicxx [-showme|-showme:compile|-showme:link] ...
+.
+.SH OPTIONS
+.TP
+--showme
+This option comes in several different variants (see below).  None of
+the variants invokes the underlying compiler; they all provide
+information on how the underlying compiler would have been invoked had
+.I --showme
+not been used.
+The basic
+.I --showme
+option outputs the command line that would be executed to compile the
+program.  \fBNOTE:\fR If a non-filename argument is passed on the
+command line, the \fI-showme\fR option will \fInot\fR display any
+additional flags.  For example, both "mpicxx --showme" and
+"mpicxx --showme my_source.c" will show all the wrapper-supplied
+flags.  But "mpicxx --showme -v" will only show the underlying
+compiler name and "-v".
+.TP
+--showme:compile
+Output the compiler flags that would have been supplied to the
+C++ compiler.
+.TP
+--showme:link
+Output the linker flags that would have been supplied to the
+C++ compiler.
+.TP
+--showme:command
+Outputs the underlying C++ compiler command (which may be one
+or more tokens).
+.TP
+--showme:incdirs
+Outputs a space-delimited (but otherwise undecorated) list of
+directories that the wrapper compiler would have provided to the
+underlying C++ compiler to indicate where relevant header files
+are located.
+.TP
+--showme:libdirs
+Outputs a space-delimited (but otherwise undecorated) list of
+directories that the wrapper compiler would have provided to the
+underlying linker to indicate where relevant libraries are located.
+.TP
+--showme:libs
+Outputs a space-delimited (but otherwise undecorated) list of library
+names that the wrapper compiler would have used to link an
+application.  For example: "mpi open-rte open-pal util".
+.TP
+--showme:version
+Outputs the version number of Open MPI.
+.TP
+--showme:help
+Output a brief usage help message.
+.PP
+See the man page for your underlying C++ compiler for other
+options that can be passed through mpicxx.
+.
+.
+.SH DESCRIPTION
+.PP
+Conceptually, the role of these commands is quite simple:
+transparently add relevant compiler and linker flags to the user's
+command line that are necessary to compile / link Open MPI
+programs, and then invoke the underlying compiler to actually perform
+the command.
+.
+.PP
+As such, these commands are frequently referred to as "wrapper"
+compilers because they do not actually compile or link applications
+themselves; they only add in command line flags and invoke the
+back-end compiler.
+.
+.
+.SS Background
+Open MPI is comprised of three software layers: OPAL (Open Portable
+Access Layer), ORTE (Open Run-Time Environment), and OMPI (Open MPI).
+There are wrapper compilers for each layer; each layer's wrapper only
+links in the libraries relevant for that layer.  Specifically, each
+layer provides the following wrapper compilers:
+.
+.TP 4
+OPAL
+\fIopalcc\fR and \fIopalc++\fR
+.
+.TP
+ORTE
+\fIortecc\fR and \fIortec++\fR
+.
+.TP
+OMPI
+\fImpicc\fR, \fImpic++\fR, \fImpicxx\fR, \fImpiCC\fR (only on systems with
+case-senstive file systems), and \fImpifort\fR (and its legacy/deprecated
+names \fImpif77\fR and \fImpif90\fR).  Note
+that \fImpic++\fR, \fImpicxx\fR, and \fImpiCC\fR all invoke the same
+underlying C++ compiler with the same options.  All are provided as
+compatibility with other MPI implementations.
+.
+.
+.SS Fortran Notes
+.PP
+The Fortran wrapper compiler for MPI (\fImpifort\fR, and its
+legacy/deprecated names \fImpif77\fR and \fImpif90\fR) can compile and
+link MPI applications that use any/all of the MPI Fortran bindings:
+.IR mpif.h ,
+the
+.I mpi
+module, and the
+.I mpi_f08
+module (assuming Open MPI was installed with support for each of these
+Fortran bindings).  Specifically: it is no longer necessary to use
+different wrapper compilers for applications that use
+.I mpif.h
+vs. applications that use the
+.I mpi
+module -- just use
+.I mpifort
+for all Fortran MPI applications.
+.
+.PP
+Note, however, that the Fortran compiler may require additional
+command-line options to enforce a specific Fortran dialect.  For
+example, in some versions of the IBM XLF compiler, if xlf90 is the
+underlying Fortran compiler,
+.IR -qfixed
+may be necessary to compile fixed-format Fortran source files.
+.
+.PP
+Finally, note that
+.I mpifort
+will be inoperative and will return an error on use if Fortran support
+was not built into the MP Ilayer.
+.
+.
+.SS Overview
+\fImpicxx\fR is a convenience wrappers for the underlying
+C++ compiler.  Translation of an Open MPI program requires the
+linkage of the Open MPI-specific libraries which may not reside in
+one of the standard search directories of ld(1).  It also often
+requires the inclusion of header files what may also not be found in a
+standard location.
+.
+.PP
+\fImpicxx\fR passes its arguments to the underlying C++
+compiler along with the -I, -L and -l options required by Open MPI
+programs.
+.
+.PP
+The Open MPI Team \fIstrongly\fR encourages using the wrapper
+compilers instead of attempting to link to the Open MPI libraries
+manually.  This allows the specific implementation of Open MPI to
+change without forcing changes to linker directives in users'
+Makefiles.  Indeed, the specific set of flags and libraries used by
+the wrapper compilers depends on how Open MPI was configured and
+built; the values can change between different installations of the
+same version of Open MPI.
+.
+.PP
+Indeed, since the wrappers are simply thin shells on top of an
+underlying compiler, there are very, very few compelling reasons
+\fInot\fR to use \fImpicxx\fR.  When it is not possible to use the
+wrappers directly, the \fI-showme:compile\fR and \fI-showme:link\fR
+options should be used to determine what flags the wrappers would have
+used.  For example:
+.
+.PP
+shell$ cc -c file1.c `mpicc -showme:compile`
+.
+.PP
+shell$ cc -c file2.c `mpicc -showme:compile`
+.
+.PP
+shell$ cc file1.o file2.o `mpicc -showme:link` -o my_mpi_program
+.
+.
+.SH NOTES
+.PP
+It is possible to make the wrapper compilers multi-lib aware.  That
+is, the libraries and includes specified may differ based on the
+compiler flags specified (for example, with the GNU compilers on
+Linux, a different library path may be used if -m32 is seen versus
+-m64 being seen).  This is not the default behavior in a standard
+build, but can be activated (for example, in a binary package
+providing both 32 and 64 bit support).  More information can be found
+at:
+.PP
+  https://github.com/open-mpi/ompi/wiki/compilerwrapper3264
+.
+.
+.SH FILES
+.PP
+The string that the wrapper compilers insert into the command line
+before invoking the underlying compiler are stored in a text file
+created by Open MPI and installed to
+\fI$pkgdata/mpicxx-wrapper-data.txt\fR, where \fI$pkgdata\fR
+is typically \fI$prefix/share/openmpi\fR, and \fI$prefix\fR is the top
+installation directory of Open MPI.
+.
+.PP
+It is rarely necessary to edit this file, but it can be examined to
+gain insight into what flags the wrappers are placing on the command
+line.
+.
+.
+.SH ENVIRONMENT VARIABLES
+.PP
+By default, the wrappers use the compilers that were selected when
+Open MPI was configured.  These compilers were either found
+automatically by Open MPI's "configure" script, or were selected by
+the user in the CC, CXX, F77, and/or FC environment variables
+before "configure" was invoked.  Additionally, other arguments
+specific to the compiler may have been selected by configure.
+.
+.PP
+These values can be selectively overridden by either editing the text
+files containing this configuration information (see the \fBFILES\fR
+section), or by setting selected environment variables of the
+form "OMPI_value".
+.
+.PP
+Valid value names are:
+.
+.TP
+CPPFLAGS
+Flags added when invoking the preprocessor (C or C++)
+.
+.TP
+LDFLAGS
+Flags added when invoking the linker (C, C++, or Fortran)
+.
+.TP
+LIBS
+Libraries added when invoking the linker (C, C++, or Fortran)
+.
+.TP
+CC
+C compiler
+.
+.TP
+CFLAGS
+C compiler flags
+.
+.TP
+CXX
+C++ compiler
+.
+.TP
+CXXFLAGS
+C++ compiler flags
+.
+.
+.TP
+FC
+Fortran compiler
+.
+.TP
+FCFLAGS
+Fortran compiler flags

macx64/mpi/openmpi/share/man/man1/mpif77.1

@@ -0,0 +1,48 @@
+.\" Copyright (c) 2008      Sun Microsystems, Inc.  All rights reserved.
+.\" Copyright (c) 2011-2012 Cisco Systems, Inc.  All rights reserved.
+.TH "mpif77,mpif90" 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.
+.SH NAME
+mpif77, mpif90 -- Deprecated Open MPI Fortran wrapper compilers
+.
+.SH SYNTAX
+mpif90 ...
+.
+.\" **************************
+.\"    Description Section
+.\" **************************
+.
+.SH DESCRIPTION
+.PP
+The
+.I mpif77
+and
+.I mpif90
+wrapper compiler names are deprecated, and will disappear in a future
+version of Open MPI.  You should use the
+.I mpifort
+wrapper compiler, instead.  While they are deprecated,
+.I mpif77
+and
+.I mpif90
+accept all the same parameters as
+.IR mpifort ,
+and behaves the same as
+.IR mpifort .
+.
+.PP
+With
+.IR mpifort ,
+you can compile any Fortran program that uses the "mpif.h", "use mpi",
+and/or "use mpi_f08" MPI Fortran interfaces.
+.
+.PP
+See mpifort(1) for more details.
+.
+.\" **************************
+.\"    See Also Section
+.\" **************************
+.
+.SH SEE ALSO
+mpifort(1)
+.

macx64/mpi/openmpi/share/man/man1/mpif90.1

@@ -0,0 +1,48 @@
+.\" Copyright (c) 2008      Sun Microsystems, Inc.  All rights reserved.
+.\" Copyright (c) 2011-2012 Cisco Systems, Inc.  All rights reserved.
+.TH "mpif77,mpif90" 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.
+.SH NAME
+mpif77, mpif90 -- Deprecated Open MPI Fortran wrapper compilers
+.
+.SH SYNTAX
+mpif90 ...
+.
+.\" **************************
+.\"    Description Section
+.\" **************************
+.
+.SH DESCRIPTION
+.PP
+The
+.I mpif77
+and
+.I mpif90
+wrapper compiler names are deprecated, and will disappear in a future
+version of Open MPI.  You should use the
+.I mpifort
+wrapper compiler, instead.  While they are deprecated,
+.I mpif77
+and
+.I mpif90
+accept all the same parameters as
+.IR mpifort ,
+and behaves the same as
+.IR mpifort .
+.
+.PP
+With
+.IR mpifort ,
+you can compile any Fortran program that uses the "mpif.h", "use mpi",
+and/or "use mpi_f08" MPI Fortran interfaces.
+.
+.PP
+See mpifort(1) for more details.
+.
+.\" **************************
+.\"    See Also Section
+.\" **************************
+.
+.SH SEE ALSO
+mpifort(1)
+.

macx64/mpi/openmpi/share/man/man1/mpifort.1

@@ -0,0 +1,265 @@
+.\" Copyright (c) 2008      Sun Microsystems, Inc.  All rights reserved.
+.\" Copyright (c) 2011-2012 Cisco Systems, Inc.  All rights reserved.
+.TH mpifort 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.
+.SH NAME
+mpifort -- Open MPI Fortran wrapper compiler
+.
+.SH SYNTAX
+mpifort [-showme|-showme:compile|-showme:link] ...
+.
+.SH OPTIONS
+.TP
+--showme
+This option comes in several different variants (see below).  None of
+the variants invokes the underlying compiler; they all provide
+information on how the underlying compiler would have been invoked had
+.I --showme
+not been used.
+The basic
+.I --showme
+option outputs the command line that would be executed to compile the
+program.  \fBNOTE:\fR If a non-filename argument is passed on the
+command line, the \fI-showme\fR option will \fInot\fR display any
+additional flags.  For example, both "mpifort --showme" and
+"mpifort --showme my_source.c" will show all the wrapper-supplied
+flags.  But "mpifort --showme -v" will only show the underlying
+compiler name and "-v".
+.TP
+--showme:compile
+Output the compiler flags that would have been supplied to the
+Fortran compiler.
+.TP
+--showme:link
+Output the linker flags that would have been supplied to the
+Fortran compiler.
+.TP
+--showme:command
+Outputs the underlying Fortran compiler command (which may be one
+or more tokens).
+.TP
+--showme:incdirs
+Outputs a space-delimited (but otherwise undecorated) list of
+directories that the wrapper compiler would have provided to the
+underlying Fortran compiler to indicate where relevant header files
+are located.
+.TP
+--showme:libdirs
+Outputs a space-delimited (but otherwise undecorated) list of
+directories that the wrapper compiler would have provided to the
+underlying linker to indicate where relevant libraries are located.
+.TP
+--showme:libs
+Outputs a space-delimited (but otherwise undecorated) list of library
+names that the wrapper compiler would have used to link an
+application.  For example: "mpi open-rte open-pal util".
+.TP
+--showme:version
+Outputs the version number of Open MPI.
+.TP
+--showme:help
+Output a brief usage help message.
+.PP
+See the man page for your underlying Fortran compiler for other
+options that can be passed through mpifort.
+.
+.
+.SH DESCRIPTION
+.PP
+Conceptually, the role of these commands is quite simple:
+transparently add relevant compiler and linker flags to the user's
+command line that are necessary to compile / link Open MPI
+programs, and then invoke the underlying compiler to actually perform
+the command.
+.
+.PP
+As such, these commands are frequently referred to as "wrapper"
+compilers because they do not actually compile or link applications
+themselves; they only add in command line flags and invoke the
+back-end compiler.
+.
+.
+.SS Background
+Open MPI is comprised of three software layers: OPAL (Open Portable
+Access Layer), ORTE (Open Run-Time Environment), and OMPI (Open MPI).
+There are wrapper compilers for each layer; each layer's wrapper only
+links in the libraries relevant for that layer.  Specifically, each
+layer provides the following wrapper compilers:
+.
+.TP 4
+OPAL
+\fIopalcc\fR and \fIopalc++\fR
+.
+.TP
+ORTE
+\fIortecc\fR and \fIortec++\fR
+.
+.TP
+OMPI
+\fImpicc\fR, \fImpic++\fR, \fImpicxx\fR, \fImpiCC\fR (only on systems with
+case-senstive file systems), and \fImpifort\fR (and its legacy/deprecated
+names \fImpif77\fR and \fImpif90\fR).  Note
+that \fImpic++\fR, \fImpicxx\fR, and \fImpiCC\fR all invoke the same
+underlying C++ compiler with the same options.  All are provided as
+compatibility with other MPI implementations.
+.
+.
+.SS Fortran Notes
+.PP
+The Fortran wrapper compiler for MPI (\fImpifort\fR, and its
+legacy/deprecated names \fImpif77\fR and \fImpif90\fR) can compile and
+link MPI applications that use any/all of the MPI Fortran bindings:
+.IR mpif.h ,
+the
+.I mpi
+module, and the
+.I mpi_f08
+module (assuming Open MPI was installed with support for each of these
+Fortran bindings).  Specifically: it is no longer necessary to use
+different wrapper compilers for applications that use
+.I mpif.h
+vs. applications that use the
+.I mpi
+module -- just use
+.I mpifort
+for all Fortran MPI applications.
+.
+.PP
+Note, however, that the Fortran compiler may require additional
+command-line options to enforce a specific Fortran dialect.  For
+example, in some versions of the IBM XLF compiler, if xlf90 is the
+underlying Fortran compiler,
+.IR -qfixed
+may be necessary to compile fixed-format Fortran source files.
+.
+.PP
+Finally, note that
+.I mpifort
+will be inoperative and will return an error on use if Fortran support
+was not built into the MP Ilayer.
+.
+.
+.SS Overview
+\fImpifort\fR is a convenience wrappers for the underlying
+Fortran compiler.  Translation of an Open MPI program requires the
+linkage of the Open MPI-specific libraries which may not reside in
+one of the standard search directories of ld(1).  It also often
+requires the inclusion of header files what may also not be found in a
+standard location.
+.
+.PP
+\fImpifort\fR passes its arguments to the underlying Fortran
+compiler along with the -I, -L and -l options required by Open MPI
+programs.
+.
+.PP
+The Open MPI Team \fIstrongly\fR encourages using the wrapper
+compilers instead of attempting to link to the Open MPI libraries
+manually.  This allows the specific implementation of Open MPI to
+change without forcing changes to linker directives in users'
+Makefiles.  Indeed, the specific set of flags and libraries used by
+the wrapper compilers depends on how Open MPI was configured and
+built; the values can change between different installations of the
+same version of Open MPI.
+.
+.PP
+Indeed, since the wrappers are simply thin shells on top of an
+underlying compiler, there are very, very few compelling reasons
+\fInot\fR to use \fImpifort\fR.  When it is not possible to use the
+wrappers directly, the \fI-showme:compile\fR and \fI-showme:link\fR
+options should be used to determine what flags the wrappers would have
+used.  For example:
+.
+.PP
+shell$ cc -c file1.c `mpicc -showme:compile`
+.
+.PP
+shell$ cc -c file2.c `mpicc -showme:compile`
+.
+.PP
+shell$ cc file1.o file2.o `mpicc -showme:link` -o my_mpi_program
+.
+.
+.SH NOTES
+.PP
+It is possible to make the wrapper compilers multi-lib aware.  That
+is, the libraries and includes specified may differ based on the
+compiler flags specified (for example, with the GNU compilers on
+Linux, a different library path may be used if -m32 is seen versus
+-m64 being seen).  This is not the default behavior in a standard
+build, but can be activated (for example, in a binary package
+providing both 32 and 64 bit support).  More information can be found
+at:
+.PP
+  https://github.com/open-mpi/ompi/wiki/compilerwrapper3264
+.
+.
+.SH FILES
+.PP
+The string that the wrapper compilers insert into the command line
+before invoking the underlying compiler are stored in a text file
+created by Open MPI and installed to
+\fI$pkgdata/mpifort-wrapper-data.txt\fR, where \fI$pkgdata\fR
+is typically \fI$prefix/share/openmpi\fR, and \fI$prefix\fR is the top
+installation directory of Open MPI.
+.
+.PP
+It is rarely necessary to edit this file, but it can be examined to
+gain insight into what flags the wrappers are placing on the command
+line.
+.
+.
+.SH ENVIRONMENT VARIABLES
+.PP
+By default, the wrappers use the compilers that were selected when
+Open MPI was configured.  These compilers were either found
+automatically by Open MPI's "configure" script, or were selected by
+the user in the CC, CXX, F77, and/or FC environment variables
+before "configure" was invoked.  Additionally, other arguments
+specific to the compiler may have been selected by configure.
+.
+.PP
+These values can be selectively overridden by either editing the text
+files containing this configuration information (see the \fBFILES\fR
+section), or by setting selected environment variables of the
+form "OMPI_value".
+.
+.PP
+Valid value names are:
+.
+.TP
+CPPFLAGS
+Flags added when invoking the preprocessor (C or C++)
+.
+.TP
+LDFLAGS
+Flags added when invoking the linker (C, C++, or Fortran)
+.
+.TP
+LIBS
+Libraries added when invoking the linker (C, C++, or Fortran)
+.
+.TP
+CC
+C compiler
+.
+.TP
+CFLAGS
+C compiler flags
+.
+.TP
+CXX
+C++ compiler
+.
+.TP
+CXXFLAGS
+C++ compiler flags
+.
+.
+.TP
+FC
+Fortran compiler
+.
+.TP
+FCFLAGS
+Fortran compiler flags

macx64/mpi/openmpi/share/man/man1/ompi-clean.1

@@ -0,0 +1,120 @@
+.\"
+.\" Copyright (c) 2004-2007 The Trustees of Indiana University and Indiana
+.\"                         University Research and Technology
+.\"                         Corporation.  All rights reserved.
+.\" Copyright 2007-2008     Sun Microsystems, Inc.
+.\"
+.TH orte-clean 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBorte-clean\fP - Cleans up any stale processes and files leftover
+from Open MPI jobs.
+
+.sp
+
+.\" **************************
+.\"    Synopsis Section
+.\" **************************
+.SH SYNOPSIS
+.ft R
+.nf
+orte-clean [--verbose]
+.br
+mpirun --pernode [--host | --hostfile \fIfile\fP] orte-clean [--verbose]
+.fi
+.sp
+
+
+.\" **************************
+.\"    Options Section
+.\" **************************
+.SH OPTIONS
+.ft R
+[-v | --verbose] This argument will run the command in verbose
+mode and print out the universes that are getting cleaned up
+as well as processes that are being killed.
+.sp
+
+.\" **************************
+.\"    Description Section
+.\" **************************
+.SH DESCRIPTION
+.ft R
+\fIorte-clean\fR attempts to clean up any processes and files
+left over from Open MPI jobs that were run in the past as well as any
+currently running jobs.  This includes OMPI infrastructure and helper
+commands, any processes that were spawned as part of the job, and any
+temporary files.  orte-clean will only act upon processes and files
+that belong to the user running the orte-clean command.  If run as
+root, it will kill off processes belonging to any users.
+.sp
+When run from the command line, orte-clean will attempt to clean up
+the local node it is run from.  When launched via mpirun, it will
+clean up the nodes selected by mpirun.
+.sp
+
+.\" **************************
+.\"    Examples Section
+.\" **************************
+.SH EXAMPLES
+.ft R
+Example 1: Clean up local node only.
+.sp
+.nf
+example% orte-clean
+.fi
+.sp
+
+Example 2: To clean up on a specific set of nodes specified on
+command line, it is recommended to use the pernode option.  This
+will run one orte-clean for each node.
+.sp
+.nf
+example% mpirun --pernode --host node1,node2,node3 orte-clean
+.fi
+.sp
+To clean up on a specific set of nodes from a file.
+.sp
+.nf
+example% mpirun --pernode --hostfile nodes_file orte-clean
+.fi
+.sp
+Example 3: Within a resource managed environment like N1GE,
+SLURM, or Torque.  The following example is from N1GE.
+.sp
+First, we see that we have two nodes with two CPUs each.
+.sp
+.nf
+example% qsh -pe orte 4
+.br
+example% mpirun -np 4 hostname
+.br
+node1
+.br
+node1
+.br
+node2
+.br
+node2
+.fi
+.sp
+Clean up all the nodes in the cluster.
+.sp
+.nf
+example% mpirun --pernode orte-clean
+.fi
+.sp
+Clean up a subset of the nodes in the cluster.
+.sp
+.nf
+example% mpirun --pernode --host node1 orte-clean
+.fi
+.sp
+
+.\" **************************
+.\"    See Also Section
+.\" **************************
+.
+.SH SEE ALSO
+.ft R
+orterun(1), orte-ps(1)
+.sp

macx64/mpi/openmpi/share/man/man1/ompi-server.1

@@ -0,0 +1,72 @@
+.\"
+.\" Copyright (c) 2007      Los Alamos National Security, LLC
+.\"                         All rights reserved.
+.\" Copyright (c) 2008-2009 Sun Microsystems, Inc.  All rights reserved.
+.\"
+.\" Man page for OMPI's ompi-server command
+.\"
+.\" .TH name     section center-footer   left-footer  center-header
+.TH OMPI-SERVER 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.\" **************************
+.\"    Name Section
+.\" **************************
+.SH NAME
+.
+ompi-server \- Server for supporting name publish/lookup operations.
+.
+.PP
+.
+.\" **************************
+.\"    Synopsis Section
+.\" **************************
+.SH SYNOPSIS
+.
+.BR ompi-server " [ options ]"
+.
+.\" **************************
+.\"    Options Section
+.\" **************************
+.SH Options
+.
+\fIompi-server\fR acts as a data server for Open MPI jobs to exchange
+contact information in support of MPI-2's Publish_name and Lookup_name
+functions.
+.
+.TP 10
+.B -h | --help
+Display help for this command
+.
+.
+.TP
+.B -d | --debug
+Enable verbose output for debugging
+.
+.
+.TP
+.B -r | --report-uri \fR<value>\fP
+Report the Open MPI contact information for the server. This information is
+required for MPI jobs to use the data server. Three parameter values are supported:
+(a) '-', indicating that the uri is to be printed to stdout; (b) '+', indicating that
+the uri is to be printed to stderr; and (c) "file:path-to-file", indicating that
+the uri is to be printed to the specified file. The "path-to-file" can be either
+absolute or relative, but must be in a location where the user has write
+permissions. Please note that the resulting file must be read-accessible to
+expected users of the server.
+.
+.
+.\" **************************
+.\"    Description Section
+.\" **************************
+.SH DESCRIPTION
+.
+.PP
+\fIompi-server\fR acts as a data server for Open MPI jobs to exchange
+contact information in support of MPI-2's Publish_name and Lookup_name
+functions.
+.
+.\" **************************
+.\"    See Also Section
+.\" **************************
+.
+.SH SEE ALSO
+.

macx64/mpi/openmpi/share/man/man1/ompi_info.1

@@ -0,0 +1,287 @@
+.\" Man page contributed by Dirk Eddelbuettel <edd@debian.org>
+.\" and released under the BSD license
+.\" Copyright (c) 2008      Sun Microsystems, Inc.  All rights reserved.
+.\" Copyright (c) 2014 Cisco Systems, Inc.  All rights reserved.
+.TH OMPI_INFO 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+ompi_info - Display information about the Open MPI installation
+.
+.\" **************************
+.\"    Synopsis Section
+.\" **************************
+.SH SYNOPSIS
+.B ompi_info [options]
+.
+.\" **************************
+.\"    Description Section
+.\" **************************
+.SH DESCRIPTION
+.PP
+.B ompi_info
+provides detailed information about the Open MPI installation.  It can
+be useful for at least three common scenarios:
+.PP
+1. Checking local configuration and seeing how Open MPI was installed.
+.PP
+2. Submitting bug reports / help requests to the Open MPI community
+(see
+.IR http://www.open-mpi.org/community/help/ )
+.PP
+3. Seeing a list of installed Open MPI plugins and querying what
+MCA parameters they support.
+.PP
+.B NOTE:
+.B ompi_info
+defaults to only showing a few MCA parameters by default (i.e., level
+1 parameters).  Use the
+.B --level
+option to enable showing more options (see the LEVELS section for more
+information).
+.
+.\" **************************
+.\"    Options Section
+.\" **************************
+.SH OPTIONS
+.B ompi_info
+accepts the following options:
+.TP 8
+.I \-a|\-\-all
+Show all configuration options and MCA parameters.  Also changes the
+default MCA parameter level to 9, unless
+.I --level
+is also specified.
+.TP 8
+.I \-\-arch
+Show architecture on which Open MPI was compiled.
+.TP 8
+.I \-c|\-\-config
+Show configuration options
+.TP 8
+.I \-gmca|\-\-gmca <param> <value>
+Pass global MCA parameters that are applicable to all contexts.
+.TP 8
+.I \-h|\-\-help
+Shows help / usage message.
+.TP 8
+.I \-\-hostname
+Show the hostname on which Open MPI was configured and built.
+.TP 8
+.I \-\-internal
+Show internal MCA parameters (not meant to be modified by users).
+.TP 8
+.I \-\-level <level>
+Show only variables with at most this level (1-9). The default is 1 unless
+\-\-all is specified without \-\-level, in which case the default is 9. See
+the LEVELS section for more information.
+.TP 8
+.I \-mca|\-\-mca <param> <value>
+Pass context-specific MCA parameters; they are considered global if --gmca is
+not used and only one context is specified.
+.TP 8
+.I \-\-param <type> <component>
+Show MCA parameters.  The first parameter is the type of the component
+to display; the second parameter is the specific component to display
+(or the keyword "all", meaning "display all components of this type").
+.TP 8
+.I \-t|\-\-type
+Show MCA parameters of the type specified in the parameter. Accepts the
+following parameters: unsigned_int, unsigned_long, unsigned_long_long,
+size_t, string, version_string, bool, double. By default level
+is 1 unless it is specified with \-\-level.
+.TP 8
+.I \-\-parsable
+When used in conjunction with other parameters, the output is
+displayed in a machine-parsable format
+.I \-\-parseable
+Synonym for --parsable
+.TP 8
+.I \-\-path <type>
+Show paths that Open MPI was configured with.  Accepts the following
+parameters: prefix, bindir, libdir, incdir, pkglibdir, sysconfdir.
+.TP 8
+.I \-\-pretty
+When used in conjunction with other parameters, the output is
+displayed in 'prettyprint' format (default)
+.TP 8
+.I \-\-selected-only
+Show only variables from selected components.
+.TP 8
+.I \-V|\-\-version
+Show version of Open MPI.
+.
+.\" **************************
+.\"    Levels Section
+.\" **************************
+.SH LEVELS
+Open MPI has many, many run-time tunable parameters (called "MCA
+parameters"), and usually only a handfull of them are useful to a
+given user.
+.
+.
+.PP
+As such, Open MPI has divided these parameters up into nine distinct
+levels, broken down into three categories, each with three
+sub-categories.
+.
+.
+.PP
+Note that since each MCA parameter is accessible through the MPI_T
+control variable API (introduced in MPI-3.0), these levels exactly
+correspond to the nine MPI_T cvar levels.
+.
+.
+.PP
+The three categories are:
+.TP 4
+.B End user
+Generally, these are parameters that are required for correctness,
+meaning that a user may need to set these just to get their MPI
+application to run correctly. For example, BTL "if_include" and
+"if_exclude" parameters fit into this category.
+.
+.TP
+.B Application tuner
+Generally, these are parameters that can be used to tweak MPI
+application performance. This even includes parameters that control
+resource exhaustion levels (e.g., number of free list entries, size of
+buffers, etc.), and could be considered "correctness" parameters if
+they're set too low. But, really -- they're tuning parameters.
+.
+.TP
+.B Open MPI developer
+Parameters in this category either don't fit in the other two, or are
+specifically intended for debugging / development of Open MPI itself.
+.
+.
+.PP
+And within each category, there are three sub-categories:
+.TP 4
+.B Basic
+This sub-category is for parameters that everyone in this category
+will want to see -- even less-advanced end users, application tuners,
+and new OMPI developers.
+.
+.TP
+.B Detailed
+This sub-category is for parameters that are generally useful, but
+users probably won't need to change them often.
+.
+.TP
+.B All
+This sub-category is for all other parameters. Such parameters are
+likely fairly esoteric.
+.
+.
+.PP
+Combining the categories and sub-categories, here's how Open MPI
+defines all nine levels:
+.TP 4
+1
+Basic information of interest to end users.
+.TP
+2
+Detailed information of interest to end users.
+.TP
+3
+All remaining information of interest to end users.
+.TP
+4
+Basic information required for application tuners.
+.TP
+5
+Detailed information required for application tuners.
+.TP
+6
+All remaining information required for application tuners.
+.TP
+7
+Basic information for Open MPI implementors.
+.TP
+8
+Detailed information for Open MPI implementors.
+.TP
+9
+All remaining information for Open MPI implementors.
+.
+.
+.PP
+By default,
+.B ompi_info
+only shows level 1 MCA parameters.  To see more MCA parameters, use
+the
+.B --level
+command line option.
+.
+.
+.PP
+.\" **************************
+.\"    Examples Section
+.\" **************************
+.SH EXAMPLES
+.TP 4
+ompi_info
+Show the default output of options and listing of installed
+components in a human-readable / prettyprint format.
+.
+.
+.TP
+ompi_info --parsable
+Show the default output of options and listing of installed
+components in a machine-parsable format.
+.
+.
+.TP
+ompi_info --param btl tcp
+Show the level 1 MCA parameters of the "tcp" BTL component in a
+human-readable / prettyprint format.
+.
+.
+.TP
+ompi_info --param btl tcp --level 6
+Show the level 1 through level 6 MCA parameters of the "tcp" BTL
+component in a human-readable / prettyprint format.
+.
+.
+.TP
+ompi_info --param btl tcp --parsable
+Show the level 1 MCA parameters of the "tcp" BTL component in a
+machine-parsable format.
+.
+.
+.TP
+ompi_info --type string --pretty-print --level 3
+Show the level 3 MCA parameters of string type in a human-readable /
+prettyprint format.
+.
+.
+.TP
+ompi_info --path bindir
+Show the "bindir" that Open MPI was configured with.
+.
+.
+.TP
+ompi_info --version
+Show the version of Open MPI version numbers in a prettyprint format.
+.
+.
+.TP
+ompi_info --all
+Show
+.I all
+information about the Open MPI installation, including all components
+that can be found, all the MCA parameters that they support (i.e.,
+levels 1 through 9), versions of Open MPI and the components, etc.
+.
+.
+.\" **************************
+.\"    Authors Section
+.\" **************************
+.SH AUTHORS
+The Open MPI maintainers -- see
+.I http://www.openmpi.org/
+or the file
+.IR AUTHORS .
+.PP
+This manual page was originally contributed by Dirk Eddelbuettel
+<edd@debian.org>, one of the Debian GNU/Linux maintainers for Open
+MPI, and may be used by others.

macx64/mpi/openmpi/share/man/man1/opal_wrapper.1

@@ -0,0 +1,78 @@
+.\" Copyright (c) 2008      Sun Microsystems, Inc.  All rights reserved.
+.\" Man page contributed by Dirk Eddelbuettel <edd@debian.org>
+.\" and released under the BSD license
+.TH OPAL_WRAPPER 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+opal_wrapper - Back-end Open MPI wrapper command
+.SH SYNOPSIS
+.B opal_wrapper [options]
+.SH DESCRIPTION
+.PP
+.B opal_wrapper
+is not meant to be called directly by end users.  It is automatically
+invoked as the back-end by the Open MPI wrapper commands such as:
+.BR mpicc ,
+.BR mpiCC ,
+.BR mpic++ ,
+and
+.BR mpifort
+(and its legacy/deprecated names
+.BR mpif77
+and
+.BR mpif90 ).
+.PP
+Some Open MPI installations may have additional wrapper commands,
+and/or have renamed the wrapper compilers listed above to avoid
+executable name conflicts with other MPI implementations.  Hence, you
+may also have wrapper compilers installed including the following
+names:
+.BR mpifort.openmpi
+(and the legacy/deprecated names
+.BR mpif90.openmpi
+and
+.BR mpif77.openmpi ),
+.BR mpicxx.openmpi ,
+.BR mpiCC.openmpi ,
+.BR mpicc.openmpi ,
+.BR mpic++.openmpi ,
+.BR opalcc ,
+.BR opalc++ ,
+.BR ortecc ,
+and
+.BR ortec++ ,
+.
+.
+.\" **************************
+.\"    See Also Section
+.\" **************************
+.SH SEE ALSO
+The following may exist depending on your particular Open MPI
+installation:
+.BR mpicc (1),
+.BR mpiCC (1),
+.BR mpic++ (1),
+.BR mpifort (1),
+.BR mpifort.openmpi (1),
+.BR mpicxx.openmpi (1),
+.BR mpiCC.openmpi (1),
+.BR mpicc.openmpi (1),
+.BR mpic++.openmpi (1),
+.BR ortecc (1),
+.BR ortec++ (1),
+.BR opalccc (1),
+and the website at
+.IR http://www.openmpi.org/ .
+.
+.
+.\" **************************
+.\"    Authors Section
+.\" **************************
+.SH AUTHORS
+The Open MPI maintainers -- see
+.I http://www.openmpi.org/
+or the file
+.IR AUTHORS .
+.PP
+This manual page was originally contributed by Dirk Eddelbuettel
+<edd@debian.org>, one of the Debian GNU/Linux maintainers for Open
+MPI, and may be used by others.

macx64/mpi/openmpi/share/man/man1/orte-clean.1

@@ -0,0 +1,120 @@
+.\"
+.\" Copyright (c) 2004-2007 The Trustees of Indiana University and Indiana
+.\"                         University Research and Technology
+.\"                         Corporation.  All rights reserved.
+.\" Copyright 2007-2008     Sun Microsystems, Inc.
+.\"
+.TH orte-clean 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBorte-clean\fP - Cleans up any stale processes and files leftover
+from Open MPI jobs.
+
+.sp
+
+.\" **************************
+.\"    Synopsis Section
+.\" **************************
+.SH SYNOPSIS
+.ft R
+.nf
+orte-clean [--verbose]
+.br
+mpirun --pernode [--host | --hostfile \fIfile\fP] orte-clean [--verbose]
+.fi
+.sp
+
+
+.\" **************************
+.\"    Options Section
+.\" **************************
+.SH OPTIONS
+.ft R
+[-v | --verbose] This argument will run the command in verbose
+mode and print out the universes that are getting cleaned up
+as well as processes that are being killed.
+.sp
+
+.\" **************************
+.\"    Description Section
+.\" **************************
+.SH DESCRIPTION
+.ft R
+\fIorte-clean\fR attempts to clean up any processes and files
+left over from Open MPI jobs that were run in the past as well as any
+currently running jobs.  This includes OMPI infrastructure and helper
+commands, any processes that were spawned as part of the job, and any
+temporary files.  orte-clean will only act upon processes and files
+that belong to the user running the orte-clean command.  If run as
+root, it will kill off processes belonging to any users.
+.sp
+When run from the command line, orte-clean will attempt to clean up
+the local node it is run from.  When launched via mpirun, it will
+clean up the nodes selected by mpirun.
+.sp
+
+.\" **************************
+.\"    Examples Section
+.\" **************************
+.SH EXAMPLES
+.ft R
+Example 1: Clean up local node only.
+.sp
+.nf
+example% orte-clean
+.fi
+.sp
+
+Example 2: To clean up on a specific set of nodes specified on
+command line, it is recommended to use the pernode option.  This
+will run one orte-clean for each node.
+.sp
+.nf
+example% mpirun --pernode --host node1,node2,node3 orte-clean
+.fi
+.sp
+To clean up on a specific set of nodes from a file.
+.sp
+.nf
+example% mpirun --pernode --hostfile nodes_file orte-clean
+.fi
+.sp
+Example 3: Within a resource managed environment like N1GE,
+SLURM, or Torque.  The following example is from N1GE.
+.sp
+First, we see that we have two nodes with two CPUs each.
+.sp
+.nf
+example% qsh -pe orte 4
+.br
+example% mpirun -np 4 hostname
+.br
+node1
+.br
+node1
+.br
+node2
+.br
+node2
+.fi
+.sp
+Clean up all the nodes in the cluster.
+.sp
+.nf
+example% mpirun --pernode orte-clean
+.fi
+.sp
+Clean up a subset of the nodes in the cluster.
+.sp
+.nf
+example% mpirun --pernode --host node1 orte-clean
+.fi
+.sp
+
+.\" **************************
+.\"    See Also Section
+.\" **************************
+.
+.SH SEE ALSO
+.ft R
+orterun(1), orte-ps(1)
+.sp

macx64/mpi/openmpi/share/man/man1/orte-info.1

@@ -0,0 +1,162 @@
+.\" Man page contributed by Dirk Eddelbuettel <edd@debian.org>
+.\" and released under the BSD license
+.\" Copyright (c) 2008      Sun Microsystems, Inc.  All rights reserved.
+.TH orte-info 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+orte-info - Display information about the ORTE installation
+.
+.\" **************************
+.\"    Synopsis Section
+.\" **************************
+.SH SYNOPSIS
+.B orte-info [options]
+.
+.\" **************************
+.\"    Description Section
+.\" **************************
+.SH DESCRIPTION
+.PP
+.B orte-info
+provides detailed information about the ORTE installation.  It can
+be useful for at least three common scenarios:
+.PP
+1. Checking local configuration and seeing how ORTE was installed.
+.PP
+2. Submitting bug reports / help requests to the ORTE community
+(see
+.IR http://www.open-mpi.org/community/help/ )
+.PP
+3. Seeing a list of installed ORTE plugins and querying what
+MCA parameters they support.
+.
+.\" **************************
+.\"    Options Section
+.\" **************************
+.SH OPTIONS
+.B orte-info
+accepts the following options:
+.TP 8
+.I \-a|\-\-all
+Show all configuration options and MCA parameters
+.TP 8
+.I \-\-arch
+Show architecture ORTE was compiled on
+.TP 8
+.I \-c|\-\-config
+Show configuration options
+.TP 8
+.I \-gmca|\-\-gmca <param> <value>
+Pass global MCA parameters that are applicable to all contexts.
+.TP 8
+.I \-h|\-\-help
+Shows help / usage message
+.TP 8
+.I \-\-hostname
+Show the hostname that ORTE was configured and built on
+.TP 8
+.I \-\-internal
+Show internal MCA parameters (not meant to be modified by users)
+.TP 8
+.I \-mca|\-\-mca <param> <value>
+Pass context-specific MCA parameters; they are considered global if --gmca is
+not used and only one context is specified.
+.TP 8
+.I \-\-param <type> <component>
+Show MCA parameters.  The first parameter is the type of the component
+to display; the second parameter is the specific component to display
+(or the keyword "all", meaning "display all components of this type").
+.TP 8
+.I \-\-parsable
+When used in conjunction with other parameters, the output is
+displayed in a machine-parsable format
+.I \-\-parseable
+Synonym for --parsable
+.TP 8
+.I \-\-path <type>
+Show paths that ORTE was configured with.  Accepts the following
+parameters: prefix, bindir, libdir, incdir, pkglibdir, sysconfdir.
+.TP 8
+.I \-\-pretty
+When used in conjunction with other parameters, the output is
+displayed in 'prettyprint' format (default)
+.TP 8
+.I \-v|\-\-version <component> <scope>
+Show version of ORTE or a component.  <component> can be the
+keywords "ompi" or "all", the name of a framework (e.g., "coll" shows
+all components in the coll framework), or the name of a specific
+component (e.g., "pls:rsh" shows the information from the rsh PLS
+component).  <scope> can be one of: full, major, minor, release,
+greek, svn.
+.
+.\" **************************
+.\"    Examples Section
+.\" **************************
+.SH EXAMPLES
+.TP 4
+orte-info
+Show the default output of options and listing of installed
+components in a human-readable / prettyprint format.
+.
+.
+.TP
+orte-info --parsable
+Show the default output of options and listing of installed
+components in a machine-parsable format.
+.
+.
+.TP
+orte-info --param rmcast udp
+Show the MCA parameters of the "udp" RMCAST component in a
+human-readable / prettyprint format.
+.
+.
+.TP
+orte-info --param rmcast udp --parsable
+Show the MCA parameters of the "udp" RMCAST component in a
+machine-parsable format.
+.
+.
+.TP
+orte-info --path bindir
+Show the "bindir" that ORTE was configured with.
+.
+.
+.TP
+orte-info --version orte full --parsable
+Show the full version numbers of ORTE (including the OPAL
+version number) in a machine-readable format.
+.
+.
+.TP
+orte-info --version rmcast major
+Show the major version number of all RMCAST components in a prettyprint
+format.
+.
+.
+.TP
+orte-info --version rmcast:tcp minor
+Show the minor version number of the TCP RMCAST component in a
+prettyprint format.
+.
+.
+.TP
+orte-info --all
+Show
+.I all
+information about the ORTE installation, including all components
+that can be found, the MCA parameters that they support, versions of
+ORTE and the components, etc.
+.
+.
+.\" **************************
+.\"    Authors Section
+.\" **************************
+.SH AUTHORS
+The ORTE maintainers -- see
+.I http://www.openmpi.org/
+or the file
+.IR AUTHORS .
+.PP
+This manual page was originally contributed by Dirk Eddelbuettel
+<edd@debian.org>, one of the Debian GNU/Linux maintainers for Open
+MPI, and may be used by others.

macx64/mpi/openmpi/share/man/man1/orte-server.1

@@ -0,0 +1,72 @@
+.\"
+.\" Copyright (c) 2007      Los Alamos National Security, LLC
+.\"                         All rights reserved.
+.\" Copyright (c) 2008-2009 Sun Microsystems, Inc.  All rights reserved.
+.\"
+.\" Man page for OMPI's ompi-server command
+.\"
+.\" .TH name     section center-footer   left-footer  center-header
+.TH OMPI-SERVER 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.\" **************************
+.\"    Name Section
+.\" **************************
+.SH NAME
+.
+ompi-server \- Server for supporting name publish/lookup operations.
+.
+.PP
+.
+.\" **************************
+.\"    Synopsis Section
+.\" **************************
+.SH SYNOPSIS
+.
+.BR ompi-server " [ options ]"
+.
+.\" **************************
+.\"    Options Section
+.\" **************************
+.SH Options
+.
+\fIompi-server\fR acts as a data server for Open MPI jobs to exchange
+contact information in support of MPI-2's Publish_name and Lookup_name
+functions.
+.
+.TP 10
+.B -h | --help
+Display help for this command
+.
+.
+.TP
+.B -d | --debug
+Enable verbose output for debugging
+.
+.
+.TP
+.B -r | --report-uri \fR<value>\fP
+Report the Open MPI contact information for the server. This information is
+required for MPI jobs to use the data server. Three parameter values are supported:
+(a) '-', indicating that the uri is to be printed to stdout; (b) '+', indicating that
+the uri is to be printed to stderr; and (c) "file:path-to-file", indicating that
+the uri is to be printed to the specified file. The "path-to-file" can be either
+absolute or relative, but must be in a location where the user has write
+permissions. Please note that the resulting file must be read-accessible to
+expected users of the server.
+.
+.
+.\" **************************
+.\"    Description Section
+.\" **************************
+.SH DESCRIPTION
+.
+.PP
+\fIompi-server\fR acts as a data server for Open MPI jobs to exchange
+contact information in support of MPI-2's Publish_name and Lookup_name
+functions.
+.
+.\" **************************
+.\"    See Also Section
+.\" **************************
+.
+.SH SEE ALSO
+.

macx64/mpi/openmpi/share/man/man1/orted.1

@@ -0,0 +1,46 @@
+.\" Man page contributed by Dirk Eddelbuettel <edd@debian.org>
+.\" and released under the BSD license.
+.\" Copyright (c) 2008      Sun Microsystems, Inc.  All rights reserved.
+.TH ORTED 1 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+orted - Start an Open RTE User-Level Daemon
+.SH SYNOPSIS
+.B orted [options]
+.SH DESCRIPTION
+.PP
+.B orted
+starts an Open RTE daemon for the Open MPI system.
+.SH NOTE
+The
+.B orted
+command is
+.I not intended to be manually invoked by end users.
+.
+It is part of the Open MPI architecture and is invoked automatically
+as necessary.  This man page is mainly intended for those adventerous
+end users and system administrators who have noticed an "orted"
+process and wondered what it is.
+.PP
+As such, the command line options accepted by the
+.B orted
+are not listed below because they are considered internal and are
+therefore subject to change between versions without warning.
+Running
+.B orted
+with the
+.I --help
+command line option will show all available options.
+.
+.
+.\" **************************
+.\"    Authors Section
+.\" **************************
+.SH AUTHORS
+The Open MPI maintainers -- see
+.I http://www.openmpi.org/
+or the file
+.IR AUTHORS .
+.PP
+This manual page was originally contributed by Dirk Eddelbuettel
+<edd@debian.org>, one of the Debian GNU/Linux maintainers for Open
+MPI, and may be used by others.

macx64/mpi/openmpi/share/man/man3/MPI.3

@@ -0,0 +1,194 @@
+.\" -*- nroff -*-
+.\" Copyright 2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+MPI \- General information Open MPI 4.0.1.
+
+.SH MPI
+.ft R
+Open MPI is an open source implementation of MPI (message-passing
+interface), the industry-standard specification for writing
+message-passing programs. Message passing is a programming model that
+gives the programmer explicit control over interprocess communication.
+.sp
+The MPI specification was developed by the MPI Forum, a group of
+software developers, computer vendors, academics, and computer-science
+researchers whose goal was to develop a standard for writing
+message-passing programs that would be efficient, flexible, and
+portable.
+.sp
+The outcome, known as the MPI Standard, was first published in 1993;
+its most recent version (MPI-2) was published in July 1997. Open MPI
+1.2 includes all MPI 1.2-compliant and MPI 2-compliant routines.
+.sp
+For more information about Open MPI, see the following URL:
+.nf
+
+   http://www.open-mpi.org
+.fi
+.sp
+The MPI standards are available at the following URL:
+.nf
+
+    http://www.mpi-forum.org
+.fi
+
+.SH MAN PAGE SYNTAX
+.ft R
+Man pages for Open MPI and Open MPI I/O routines are named according to C syntax, that is, they begin with the prefix "MPI_", all in uppercase, and the first letter following the "MPI_" prefix is also uppercase. The rest of the letters in the routine are all lowercase, for example, "MPI_Comm_get_attr".
+
+.SH ENVIRONMENT
+.ft R
+To fine-tune your Open MPI environment, you can either use arguments to the \fImpirun\fP, \fIorterun\fP, or \fImpiexec\fP commands, or you can use MCA parameters.
+.sp
+For more information on arguments, see the orterun.1 man page.
+.sp
+For a complete listing of MCA parameters and their descriptions, issue the command \fIompi_info -h\fP or \fIompi_info -param all all\fP. This information also appears in the FAQ on the Open MPI web site at:
+.nf
+
+   http://www.open-mpi.org/faq/?category=tuning#mca-params
+.fi
+
+.SH ERRORS
+.ft R
+All MPI routines (except MPI_Wtime and MPI_Wtick) return an error value; C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does not guarantee that an MPI program can continue past
+an error.
+.sp
+For more information on Open MPI error codes, see \fImpi.h\fP in the \fIinclude\fP directory.
+.sp
+Standard error return classes for Open MPI:
+.sp
+.nf
+MPI_SUCCESS                0      Successful return code.
+MPI_ERR_BUFFER             1      Invalid buffer pointer.
+MPI_ERR_COUNT              2      Invalid count argument.
+MPI_ERR_TYPE               3      Invalid datatype argument.
+MPI_ERR_TAG                4      Invalid tag argument.
+MPI_ERR_COMM               5      Invalid communicator.
+
+MPI_ERR_RANK               6      Invalid rank.
+MPI_ERR_REQUEST            7      Invalid MPI_Request handle.
+MPI_ERR_ROOT               8      Invalid root.
+MPI_ERR_GROUP              9      Null group passed to function.
+MPI_ERR_OP                10      Invalid operation.
+
+MPI_ERR_TOPOLOGY          11      Invalid topology.
+MPI_ERR_DIMS              12      Illegal dimension argument.
+MPI_ERR_ARG               13      Invalid argument.
+MPI_ERR_UNKNOWN           14      Unknown error.
+MPI_ERR_TRUNCATE          15      Message truncated on receive.
+
+MPI_ERR_OTHER             16      Other error; use Error_string.
+MPI_ERR_INTERN            17      Internal error code.
+MPI_ERR_IN_STATUS         18      Look in status for error value.
+MPI_ERR_PENDING           19      Pending request.
+MPI_ERR_ACCESS            20      Permission denied.
+
+MPI_ERR_AMODE             21      Unsupported amode passed to open.
+MPI_ERR_ASSERT            22      Invalid assert.
+MPI_ERR_BAD_FILE          23      Invalid file name (for example,
+                                  path name too long).
+MPI_ERR_BASE              24      Invalid base.
+MPI_ERR_CONVERSION        25      An error occurred in a user-supplied
+                                  data-conversion function.
+
+MPI_ERR_DISP              26      Invalid displacement.
+MPI_ERR_DUP_DATAREP       27      Conversion functions could not
+                                  be registered because a data
+                                  representation identifier that was
+                                  already defined was passed to
+                                  MPI_REGISTER_DATAREP.
+MPI_ERR_FILE_EXISTS       28      File exists.
+MPI_ERR_FILE_IN_USE       29      File operation could not be
+                                  completed, as the file is currently
+                                  open by some process.
+MPI_ERR_FILE              30      Invalid file handle.
+
+MPI_ERR_INFO_KEY          31      Illegal info key.
+MPI_ERR_INFO_NOKEY        32      No such key.
+MPI_ERR_INFO_VALUE        33      Illegal info value.
+MPI_ERR_INFO              34      Invalid info object.
+MPI_ERR_IO                35      I/O error.
+
+MPI_ERR_KEYVAL            36      Illegal key value.
+MPI_ERR_LOCKTYPE          37      Invalid locktype.
+MPI_ERR_NAME              38      Name not found.
+MPI_ERR_NO_MEM            39      Memory exhausted.
+MPI_ERR_NOT_SAME          40      Collective argument not identical
+                                  on all processes, or collective
+                                  routines called in a different order
+                                  by different processes.
+
+MPI_ERR_NO_SPACE          41      Not enough space.
+MPI_ERR_NO_SUCH_FILE      42      File (or directory) does not exist.
+MPI_ERR_PORT              43      Invalid port.
+MPI_ERR_QUOTA             44      Quota exceeded.
+MPI_ERR_READ_ONLY         45      Read-only file system.
+
+MPI_ERR_RMA_CONFLICT      46      Conflicting accesses to window.
+MPI_ERR_RMA_SYNC          47      Erroneous RMA synchronization.
+MPI_ERR_SERVICE           48      Invalid publish/unpublish.
+MPI_ERR_SIZE              49      Invalid size.
+MPI_ERR_SPAWN             50      Error spawning.
+
+MPI_ERR_UNSUPPORTED_DATAREP
+                          51      Unsupported datarep passed to
+                                  MPI_File_set_view.
+MPI_ERR_UNSUPPORTED_OPERATION
+                          52      Unsupported operation, such as
+                                  seeking on a file that supports
+                                  only sequential access.
+MPI_ERR_WIN               53      Invalid window.
+MPI_T_ERR_MEMORY          54      Out of memory.
+MPI_T_ERR_NOT_INITIALIZED 55      Interface not initialized.
+
+MPI_T_ERR_CANNOT_INIT     56      Interface not in the state to be
+                                  initialized.
+MPI_T_ERR_INVALID_INDEX   57      The enumeration index is invalid.
+MPI_T_ERR_INVALID_ITEM    58      The item index queried is out of
+                                  range.
+MPI_T_ERR_INVALID_HANDLE  59      The handle is invalid.
+MPI_T_ERR_OUT_OF_HANDLES  60      No more handles available.
+
+MPI_T_ERR_OUT_OF_SESSIONS 61      No more sessions available.
+MPI_T_ERR_INVALID_SESSION 62      Session argument is not a valid
+                                  session.
+MPI_T_ERR_CVAR_SET_NOT_NOW
+                          63      Variable cannot be set at this
+                                  moment.
+MPI_T_ERR_CVAR_SET_NEVER  64      Variable cannot be set until end of
+                                  execution.
+MPI_T_ERR_PVAR_NO_STARTSTOP
+                          65      Variable cannot be started or stopped.
+
+MPI_T_ERR_PVAR_NO_WRITE   66      Variable cannot be written or reset.
+MPI_T_ERR_PVAR_NO_ATOMIC  67      Variable cannot be read and written
+                                  atomically.
+MPI_ERR_RMA_RANGE         68      Target memory is not part of the
+                                  window (in the case of a window
+                                  created with MPI_WIN_CREATE_DYNAMIC,
+                                  target memory is not attached).
+MPI_ERR_RMA_ATTACH        69      Memory cannot be attached (e.g.,
+                                  because of resource exhaustion).
+MPI_ERR_RMA_FLAVOR        70      Passed window has the wrong flavor
+                                  for the called function.
+
+MPI_ERR_RMA_SHARED        71      Memory cannot be shared (e.g., some
+                                  process in the group of the specified
+                                  communicator cannot expose shared
+                                  memory).
+MPI_T_ERR_INVALID         72      Invalid use of the interface or bad
+                                  parameter values(s).
+MPI_T_ERR_INVALID_NAME    73      The variable or category name is
+                                  invalid.
+
+MPI_ERR_LASTCODE          93      Last error code.
+.fi
+

macx64/mpi/openmpi/share/man/man3/MPIX_Allgather_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Allgatherv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Allreduce_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Alltoall_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Alltoallv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Alltoallw_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Barrier_init.3

@@ -0,0 +1,156 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2018      FUJITSU LIMITED.  All rights reserved.
+.\" $COPYRIGHT$
+.TH MPIX_Barrier_init 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPIX_Allgather_init, MPIX_Allgatherv_init, MPIX_Allreduce_init, MPIX_Alltoall_init, MPIX_Alltoallv_init, MPIX_Alltoallw_init, MPIX_Barrier_init, MPIX_Bcast_init, MPIX_Exscan_init, MPIX_Gather_init, MPIX_Gatherv_init, MPIX_Reduce_init, MPIX_Reduce_scatter_init, MPIX_Reduce_scatter_block_init, MPIX_Scan_init, MPIX_Scatter_init, MPIX_Scatterv_init, MPIX_Neighbor_allgather_init, MPIX_Neighbor_allgatherv_init, MPIX_Neighbor_alltoall_init, MPIX_Neighbor_alltoallv_init, MPIX_Neighbor_alltoallw_init\fP \- Builds a handle for a collective communication or neighborhood collective communication
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+#include <mpi-ext.h>
+int MPIX_Allgather_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Allgatherv_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[],
+	const int \fIdispls\fP[], MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Allreduce_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP, int \fIcount\fP,
+	MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Alltoall_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Alltoallv_init(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP,
+	const int \fIrecvcounts\fP[], const int \fIrdispls\fP[], MPI_Datatype \fIrecvtype\fP,
+	MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Alltoallw_init(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], const MPI_Datatype \fIsendtypes\fP[], void *\fIrecvbuf\fP,
+	const int \fIrecvcounts\fP[], const int \fIrdispls\fP[],
+	const MPI_Datatype \fIrecvtypes\fP[], MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Barrier_init(MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Bcast_init(void *\fIbuffer\fP, int \fIcount\fP, MPI_Datatype \fIdatatype\fP,
+	int \fIroot\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Exscan_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP, int \fIcount\fP,
+	MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Gather_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Gatherv_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[],
+	const int \fIdispls\fP[], MPI_Datatype \fIrecvtype\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Reduce_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP, int \fIcount\fP,
+	MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Reduce_scatter_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP,
+	const int \fIrecvcounts\fP[], MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP,
+	MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Reduce_scatter_block_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP,
+	int \fIrecvcount\fP, MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Scan_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP, int \fIcount\fP,
+	MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Scatter_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Scatterv_init(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIdispls\fP[], MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP,
+	int \fIrecvcount\fP, MPI_Datatype \fIrecvtype\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Neighbor_allgather_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Neighbor_allgatherv_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[],
+	const int \fIdispls\fP[], MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Neighbor_alltoall_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Neighbor_alltoallv_init(const void *\fIsendbuf\fP,
+	const int \fIsendcounts\fP[], const int \fIsdispls\fP[], MPI_Datatype \fIsendtype\fP,
+	void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[], const int \fIrdispls\fP[],
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Neighbor_alltoallw_init(const void *\fIsendbuf\fP,
+	const int \fIsendcounts\fP[], const MPI_Aint \fIsdispls\fP[],
+	const MPI_Datatype \fIsendtypes\fP[], void *\fIrecvbuf\fP,
+	const int \fIrecvcounts\fP[], const MPI_Aint \fIrdispls\fP[],
+	const MPI_Datatype \fIrecvtypes\fP[], MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+.fi
+.SH DESCRIPTION
+.ft R
+Creates a persistent communication request for a collective operation or neighborhood collective operation.
+
+As of Nov. 2018, the feature of persistent collective communication operations and persistent collective neighborhood communication operations is planned to be included in the next MPI Standard after MPI-3.1.
+.nf
+
+    https://github.com/mpi-forum/mpi-issues/issues/25
+.fi
+
+Open MPI implements 2018 Draft Specification of the MPI standard shown in the following URL.
+.nf
+
+    https://www.mpi-forum.org/docs/drafts/mpi-2018-draft-report.pdf
+.fi
+
+The interface may still change in the standard. Therefore the prefix \fIMPIX_\fP is used instead of \fIMPI_\fP for these request creation routines. To start, complete, and free the created request, usual MPI routines (\fIMPI_Start\fP etc.) can be used.
+
+Future versions of Open MPI will switch to the \fIMPI_\fP prefix and will not require the C header file \fImpi-ext.h\fP, the Fortran modules \fImpi_ext\fP and \fImpi_f08_ext\fP, and the Fortran header file \fImpif-ext.h\fP once the MPI Standard which includes this feature is published.
+
+.SH EXAMPLE
+.nf
+
+    MPI_Request req;
+    MPIX_Barrier_init(MPI_COMM_WORLD, MPI_INFO_NULL, &req);
+    MPI_Start(&req);
+    MPI_Wait(&req, MPI_STATUS_IGNORE);
+    MPI_Request_free(&req);
+.fi
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Start
+.br
+MPI_Startall
+.br
+MPI_Request_free

macx64/mpi/openmpi/share/man/man3/MPIX_Bcast_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Exscan_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Gather_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Gatherv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Neighbor_allgather_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Neighbor_allgatherv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Neighbor_alltoall_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Neighbor_alltoallv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Neighbor_alltoallw_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Query_cuda_support.3

@@ -0,0 +1,28 @@
+.\" Copyright 2007-2010 Oracle and/or its affiliates.  All rights reserved.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" Copyright (c) 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright (c) 2015 NVIDIA, Inc. All rights reserved.
+.TH MPIx_CUDA_SUPPORT 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPIX_Query_cuda_support\fP \- Returns 1 if there is CUDA aware support and 0 if there is not.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+#include <mpi-ext.h>
+
+int MPIX_Query_cuda_support(void)
+.fi
+.SH DESCRIPTION
+.ft R
+
+.SH Examples
+.ft R
+
+.SH See Also
+.ft R
+.nf
+
+.fi

macx64/mpi/openmpi/share/man/man3/MPIX_Reduce_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Reduce_scatter_block_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Reduce_scatter_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Scan_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Scatter_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Scatterv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPI_Abort.3

@@ -0,0 +1,51 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Abort 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Abort\fP \- Terminates MPI execution environment.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Abort(MPI_Comm \fIcomm\fP, int\fI errorcode\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator of tasks to abort.
+.TP 1i
+errorcode
+Error code to return to invoking environment.
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This routine makes a "best attempt" to abort all tasks in the group of
+comm. This function does not require that the invoking environment take any
+action with the error code. However, a UNIX or POSIX
+environment should handle this as a return errorcode from the main program
+or an abort (errorcode).
+.sp
+The long-term goal of the Open MPI implementation is to terminate all processes in all tasks that contain a process in \fIcomm\fP, and the error code is not returned to the invoking environment. At the moment, this isn't fully implemented and MPI_Abort will terminate the entire job.
+.sp
+Note: All associated processes are sent a SIGTERM.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Accumulate.3

@@ -0,0 +1,119 @@
+.\" -*- nroff -*-
+.\" Copyright 2013-2014 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Accumulate 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Accumulate\fP, \fBMPI_Raccumulate\fP \- Combines the contents of the origin buffer with that of a target buffer.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Accumulate(const void *\fIorigin_addr\fP, int \fIorigin_count\fP,
+	MPI_Datatype \fIorigin_datatype\fP, int \fItarget_rank\fP,
+	MPI_Aint \fItarget_disp\fP, int \fItarget_count\fP,
+	MPI_Datatype \fItarget_datatype\fP, MPI_Op \fIop\fP, MPI_Win \fIwin\fP)
+
+int MPI_Raccumulate(const void *\fIorigin_addr\fP, int \fIorigin_count\fP,
+	MPI_Datatype \fIorigin_datatype\fP, int \fItarget_rank\fP,
+	MPI_Aint \fItarget_disp\fP, int \fItarget_count\fP,
+	MPI_Datatype \fItarget_datatype\fP, MPI_Op \fIop\fP, MPI_Win \fIwin\fP,
+	MPI_Request *\fIrequest\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+origin_addr
+Initial address of buffer (choice).
+.ft R
+.TP 1i
+origin_count
+Number of entries in buffer (nonnegative integer).
+.ft R
+.TP 1i
+origin_datatype
+Data type of each buffer entry (handle).
+.ft R
+.TP 1i
+target_rank
+Rank of target (nonnegative integer).
+.ft R
+.TP 1i
+target_disp
+Displacement from start of window to beginning of target buffer (nonnegative integer).
+.ft R
+.TP 1i
+target_count
+Number of entries in target buffer (nonnegative integer).
+.ft R
+.TP 1i
+target_datatype
+Data type of each entry in target buffer (handle).
+.ft R
+.TP 1i
+op
+Reduce operation (handle).
+.ft R
+.TP 1i
+win
+Window object (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+MPI_Raccumulate: RMA request
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+\fBMPI_Accumulate\fP is a function used for one-sided MPI communication that adds the contents of the origin buffer (as defined by \fIorigin_addr\fP, \fIorigin_count\fP, and \fIorigin_datatype\fP) to the buffer specified by the arguments \fItarget_count\fP and \fItarget_datatype\fP, at offset \fItarget_disp\fP, in the target window specified by \fItarget_rank\fP and \fIwin\fP, using the operation \fIop\fP. The target window can only be accessed by processes within the same node. This is similar to MPI_Put, except that data is combined into the target area instead of overwriting it.
+.sp
+Any of the predefined operations for MPI_Reduce can be used. User-defined functions cannot be used. For example, if \fIop\fP is MPI_SUM, each element of the origin buffer is added to the corresponding element in the target, replacing the former value in the target.
+.sp
+Each datatype argument must be a predefined data type or a derived data type, where all basic components are of the same predefined data type. Both datatype arguments must be constructed from the same predefined data type. The operation \fIop\fP applies to elements of that predefined type. The \fItarget_datatype\fP argument must not specify overlapping entries, and the target buffer must fit in the target window.
+.sp
+A new predefined operation, MPI_REPLACE, is defined. It corresponds to the associative function f(a, b) =b; that is, the current value in the target memory is replaced by the value supplied by the origin.
+.sp
+\fBMPI_Raccumulate\fP is similar to \fBMPI_Accumulate\fP, except that it allocates a communication request object and associates it with the request handle (the argument \fIrequest\fP) that can be used to wait or test for completion. The completion of an \fBMPI_Raccumulate\fP operation indicates that the \fIorigin_addr\fP buffer is free to be updated. It does not indicate that the operation has completed at the target window.
+
+.SH FORTRAN 77 NOTES
+.ft R
+The MPI standard prescribes portable Fortran syntax for
+the \fITARGET_DISP\fP argument only for Fortran 90.  FORTRAN 77
+users may use the non-portable syntax
+.sp
+.nf
+     INTEGER*MPI_ADDRESS_KIND \fITARGET_DISP\fP
+.fi
+.sp
+where MPI_ADDRESS_KIND is a constant defined in mpif.h
+and gives the length of the declared integer in bytes.
+
+.SH NOTES
+MPI_Put is a special case of \fBMPI_Accumulate\fP, with the operation MPI_REPLACE. Note, however, that MPI_Put and \fBMPI_Accumulate\fP have different constraints on concurrent updates.
+.sp
+It is the user's responsibility to guarantee that, when
+using the accumulate functions, the target displacement argument is such
+that accesses to the window are properly aligned according to the data
+type arguments in the call to the \fBMPI_Accumulate\fP function.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Put
+MPI_Get_accumulate
+MPI_Reduce

macx64/mpi/openmpi/share/man/man3/MPI_Add_error_class.3

@@ -0,0 +1,77 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Add_error_class 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+.nf
+\fBMPI_Add_error_class\fP \- Creates a new error class and returns its value
+
+.fi
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Add_error_class(int *\fIerrorclass\fP)
+
+.fi
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.4i
+errorclass
+New error class (integer).
+.ft R
+.TP 1.4i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+The function MPI_Add_error_class creates a new, local error
+class.
+
+.SH NOTES
+.ft R
+Because this function is local, the same value of \fIerrorclass\fP may
+not be returned on all processes that make this call, even if they
+call the function concurrently. Thus, same error on different
+processes may not cause the same value of \fIerrorclass\fP to be
+returned. To reduce the potential for confusion, MPI_Add_error_string
+may be used on multiple processes to associate the same error string
+with the newly created \fIerrorclass\fP. Even though \fIerrorclass\fP
+may not be consistent across processes, using MPI_Add_error_string
+will ensure the error string associated with it will be the same
+everywhere.
+.sp
+No function is provided to free error classes, as it is not expected
+that an application will create them in significant numbers.
+.sp
+The value returned is always greater than or equal to MPI_ERR_LASTCODE.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Add_error_code
+MPI_Add_error_string
+MPI_Error_class
+MPI_Error_string
+

macx64/mpi/openmpi/share/man/man3/MPI_Add_error_code.3

@@ -0,0 +1,69 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Add_error_code 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+\fBMPI_Add_error_code\fP \- Creates a new error code associated
+with \fIerrorclass\fP
+
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Add_error_code(int \fIerrorclass\fP, int *\fIerrorcode\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1.4i
+errorclass
+MPI error class (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.4i
+errorcode
+Error code returned by an MPI routine or an MPI error class (integer).
+.ft R
+.TP 1.4i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+Creates a new error code associated with \fIerrorclass\fP and returns
+its value in \fIerrorcode\fP.
+
+.SH NOTES
+.ft R
+No function is provided to free error codes, as it is not expected
+that an application will create them in significant numbers.
+.sp
+The value returned is always greater than or equal to MPI_ERR_LASTCODE.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Add_error_class
+MPI_Error_class
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Add_error_string.3

@@ -0,0 +1,72 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Add_error_string 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+.nf
+\fBMPI_Add_error_string\fP \- Associates a string with an error code or class
+
+.fi
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Add_error_string(int \fIerrorcode\fP, const char *\fIstring\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1.4i
+errorcode
+MPI error class, or an error code returned by an MPI routine (integer).
+.ft R
+.TP 1.4i
+string
+Text that corresponds to the error code or class (string).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1.4i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This routine associates an error string with an error code or
+class. Calling MPI_Add_error_string for an error code or class that
+already has an associated error string will replace the old string
+with the new one. It is erroneous to call MPI_Add_error_string for an
+error value not generated via MPI_Add_error_class or
+MPI_Add_error_code (e.g., an error code or class with a value not
+greater than MPI_LAST_ERRCODE).
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Add_error_class
+MPI_Add_error_code
+MPI_Error_class
+MPI_Error_string
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Address.3

@@ -0,0 +1,84 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Address 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Address\fP \- Gets the address of a location in memory -- use of this routine is deprecated.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Address(void *\fIlocation\fP, MPI_Aint\fP *address\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+location
+Location in caller memory (choice).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+address
+Address of location (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Note that use of this routine is \fIdeprecated\fP as of MPI-2. Please use MPI_Get_address instead.
+.sp
+This deprecated routine is not available in C++.
+.sp
+The address of a location in memory can be found by invoking this function. Returns the (byte) address of location.
+.sp
+Example: Using MPI_Address for an array.
+.sp
+.nf
+REAL A(100,100)
+.fi
+.br
+   INTEGER I1, I2, DIFF
+.br
+   CALL MPI_ADDRESS(A(1,1), I1, IERROR)
+.br
+   CALL MPI_ADDRESS(A(10,10), I2, IERROR)
+.br
+   DIFF = I2 - I1
+.br
+! The value of DIFF is 909*sizeofreal; the values of I1 and I2 are
+.br
+! implementation dependent.
+.fi
+
+.SH NOTES
+.ft R
+This routine is provided for both Fortran and C programmers and may be useful when writing portable code. In the current release, the address returned by this routine will be the same as that produced by the C & operator.
+.sp
+C users may be tempted to avoid using
+MPI_Address and rely on the availability of the address operator &.
+Note, however, that & cast-expression is a pointer, not an address.
+ANSI C does not require that the value of a pointer (or the pointer
+cast to int) be the absolute address of the object pointed at although
+this is commonly the case. Furthermore, referencing may not have a unique
+definition on machines with a segmented address space. The use of
+MPI_Address to "reference" C variables guarantees portability to
+such machines as well.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+MPI_Get_address
+.br

macx64/mpi/openmpi/share/man/man3/MPI_Aint_add.3

@@ -0,0 +1,73 @@
+.\" -*- nroff -*-
+.\" Copyright 2013-2015 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Aint_add 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Aint_add\fP, \fBMPI_Aint_diff\fP \- Portable functions for
+arithmetic on MPI_Aint values.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+MPI_Aint MPI_Aint_add(MPI_Aint \fIbase\fP, MPI_Aint \fIdisp\fP)
+
+MPI_Aint MPI_Aint_diff(MPI_Aint \fIaddr1\fP, MPI_Aint \fIaddr2\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+base
+Base address (integer).
+.ft R
+.TP 1i
+disp
+Displacement (integer).
+.ft R
+.TP 1i
+addr1
+Minuend address (integer).
+.ft R
+.TP
+addr2
+Subtrahend address (integer).
+
+.SH DESCRIPTION
+.ft R
+\fBMPI_Aint_add\fP produces a new MPI_Aint value that is equivalent to the sum of
+the \fIbase\fP and \fIdisp\fP arguments, where \fIbase\fP represents
+a base address returned by a call to \fBMPI_Get_address\fP and
+\fIdisp\fP represents a signed integer displacement. The resulting
+address is valid only at the process that generated \fIbase\fP, and it
+must correspond to a location in the same object referenced by
+\fIbase\fP, as described in MPI-3.1 \[char167] 4.1.12. The addition is
+performed in a manner that results in the correct MPI_Aint
+representation of the output address, as if the process that
+originally produced \fIbase\fP had called:
+
+.nf
+        \fBMPI_Get_address\fP ((char *) \fIbase\fP + \fIdisp\fP, &\fIresult\fP);
+.fi
+.sp
+.ft R
+\fBMPI_Aint_diff\fP produces a new MPI_Aint value that is equivalent
+to the difference between \fIaddr1\fP and \fIaddr2\fP arguments, where
+\fIaddr1\fP and \fIaddr2\fP represent addresses returned by calls to
+\fBMPI_Get_address\fP. The resulting address is valid only at the
+process that generated \fIaddr1\fP and \fIaddr2\fP, and \fIaddr1\fP
+and \fIaddr2\fP must correspond to locations in the same object in the
+same process, as described in MPI-3.1 \[char167] 4.1.12. The difference is
+calculated in a manner that results in the signed difference from
+\fIaddr1\fP to \fIaddr2\fP, as if the process that originally produced
+the addresses had called (char *) \fIaddr1\fP - (char *) \fIaddr2\fP
+on the addresses initially passed to \fBMPI_Get_address\fP.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Get_address

macx64/mpi/openmpi/share/man/man3/MPI_Aint_diff.3

@@ -0,0 +1 @@
+.so man3/MPI_Aint_add.3

macx64/mpi/openmpi/share/man/man3/MPI_Allgather.3

@@ -0,0 +1,129 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Allgather 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Allgather, MPI_Iallgather\fP \- Gathers data from all processes and distributes it to all processes
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Allgather(const void\fI *sendbuf\fP, int \fI sendcount\fP,
+	 MPI_Datatype\fI sendtype\fP, void\fI *recvbuf\fP, int\fI recvcount\fP,
+	 MPI_Datatype\fI recvtype\fP, MPI_Comm\fI comm\fP)
+
+int MPI_Iallgather(const void\fI *sendbuf\fP, int \fI sendcount\fP,
+	 MPI_Datatype\fI sendtype\fP, void\fI *recvbuf\fP, int\fI recvcount\fP,
+	 MPI_Datatype\fI recvtype\fP, MPI_Comm\fI comm\fP, MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+sendbuf
+Starting address of send buffer (choice).
+.TP 1i
+sendcount
+Number of elements in send buffer (integer).
+.TP 1i
+sendtype
+Datatype of send buffer elements (handle).
+.TP 1i
+recvbuf
+Starting address of recv buffer (choice).
+.TP 1i
+recvcount
+Number of elements received from any process (integer).
+.TP 1i
+recvtype
+Datatype of receive buffer elements (handle).
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+recvbuf
+Address of receive buffer (choice).
+.ft R
+.TP 1i
+request
+Request (handle, non-blocking only).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Allgather is similar to MPI_Gather, except that all processes receive the result, instead of just the root. In other words, all processes contribute to the result, and all processes receive the result.
+.sp
+The type signature associated with sendcount, sendtype at a process must be equal to the type signature associated with recvcount, recvtype at any other process.
+.sp
+The outcome of a call to MPI_Allgather(\&...) is as if all processes executed n calls to
+.sp
+.nf
+  MPI_Gather(sendbuf,sendcount,sendtype,recvbuf,recvcount,
+             recvtype,root,comm),
+.fi
+.sp
+.fi
+for root = 0 , ..., n-1. The rules for correct usage of MPI_Allgather are easily found from the corresponding rules for MPI_Gather.
+.sp
+\fBExample:\fR The all-gather version of Example 1 in MPI_Gather. Using  MPI_Allgather, we will gather 100 ints from every process in the group to every process.
+.sp
+.nf
+MPI_Comm comm;
+    int gsize,sendarray[100];
+    int *rbuf;
+    \&...
+    MPI_Comm_size( comm, &gsize);
+    rbuf = (int *)malloc(gsize*100*sizeof(int));
+    MPI_Allgather( sendarray, 100, MPI_INT, rbuf, 100, MPI_INT, comm);
+.fi
+.sp
+After the call, every process has the group-wide concatenation of the sets of data.
+
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-gather operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR.  In this case, \fIsendcount\fR and \fIsendtype\fR are ignored.  The input data of each process is assumed to be in the area where that process would receive its own contribution to the receive buffer.  Specifically, the outcome of a call to MPI_Allgather that used the in-place option is identical to the case in which all processes executed \fIn\fR calls to
+.sp
+.nf
+   MPI_GATHER ( MPI_IN_PLACE, 0, MPI_DATATYPE_NULL, recvbuf,
+   recvcount, recvtype, root, comm )
+
+for root =0, ... , n-1.
+.fi
+.sp
+Note that MPI_IN_PLACE is a special kind of value; it has the same restrictions on its use as MPI_BOTTOM.
+.sp
+Because the in-place option converts the receive buffer into a send-and-receive buffer, a Fortran binding that includes INTENT must mark these as INOUT, not OUT.
+.sp
+.SH WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the gather operation occurs in two phases.  The data is gathered from all the members of the first group and received by all the members of the second group.  Then the data is gathered from all the members of the second group and received by all the members of the first.  The operation, however, need not be symmetric.  The number of items sent by the processes in first group need not be equal to the number of items sent by the the processes in the second group.  You can move data in only one direction by giving \fIsendcount\fR a value of 0 for communication in the reverse direction.
+.sp
+The first group defines the root process.  The root process uses MPI_ROOT as the value of \fIroot\fR.  All other processes in the first group use MPI_PROC_NULL as the value of \fIroot\fR.  All processes in the second group use the rank of the root process in the first group as the value of \fIroot\fR.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+.sp
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Allgatherv
+.br
+MPI_Gather
+

macx64/mpi/openmpi/share/man/man3/MPI_Allgatherv.3

@@ -0,0 +1,113 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2007-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Allgatherv 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Allgatherv, MPI_Iallgatherv\fP \- Gathers data from all processes and delivers it to all. Each process may contribute a different amount of data.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Allgatherv(const void\fI *sendbuf\fP, int\fI sendcount\fP,
+	MPI_Datatype\fI sendtype\fP, void\fI *recvbuf\fP, const int\fI recvcounts[]\fP,
+	const int\fI displs[]\fP, MPI_Datatype\fI recvtype\fP, MPI_Comm\fI comm\fP)
+
+int MPI_Iallgatherv(const void\fI *sendbuf\fP, int\fI sendcount\fP,
+	MPI_Datatype\fI sendtype\fP, void\fI *recvbuf\fP, const int\fI recvcounts[]\fP,
+	const int\fI displs[]\fP, MPI_Datatype\fI recvtype\fP, MPI_Comm\fI comm\fP,
+        MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+sendbuf
+Starting address of send buffer (choice).
+.TP 1i
+sendcount
+Number of elements in send buffer (integer).
+.TP 1i
+sendtype
+Datatype of send buffer elements (handle).
+.TP 1i
+recvcount
+Integer array (of length group size) containing the number of elements that are received from each process.
+.TP 1i
+displs
+Integer array (of length group size). Entry i specifies the displacement (relative to recvbuf) at which to place the incoming data from process i.
+.TP 1i
+recvtype
+Datatype of receive buffer elements (handle).
+.TP 1i
+comm
+Communicator (handle).
+.sp
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+recvbuf
+Address of receive buffer (choice).
+.TP 1i
+request
+Request (handle, non-blocking only).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Allgatherv is similar to MPI_Allgather in that all processes gather data from all other processes, except that each process can send a different amount of data. The block of data sent from the jth process is received by every process and placed in the jth block of the buffer
+.I recvbuf.
+.sp
+The type signature associated with sendcount, sendtype, at process j must be equal to the type signature associated with recvcounts[j], recvtype at any other process.
+.sp
+The outcome is as if all processes executed calls to
+.nf
+MPI_Gatherv(sendbuf,sendcount,sendtype,recvbuf,recvcount,
+            displs,recvtype,root,comm)
+.fi
+.sp
+for root = 0 , ..., n-1. The rules for correct usage of MPI_Allgatherv are easily found from the corresponding rules for MPI_Gatherv.
+
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-gather operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR.  In this case, \fIsendcount\fR and \fIsendtype\fR are ignored.  The input data of each process is assumed to be in the area where that process would receive its own contribution to the receive buffer.  Specifically, the outcome of a call to MPI_Allgather that used the in-place option is identical to the case in which all processes executed \fIn\fR calls to
+.sp
+.nf
+   MPI_GATHERV ( MPI_IN_PLACE, 0, MPI_DATATYPE_NULL, recvbuf,
+   recvcounts, displs, recvtype, root, comm )
+
+for root =0, ... , n-1.
+.fi
+.sp
+Note that MPI_IN_PLACE is a special kind of value; it has the same restrictions on its use as MPI_BOTTOM.
+.sp
+Because the in-place option converts the receive buffer into a send-and-receive buffer, a Fortran binding that includes INTENT must mark these as INOUT, not OUT.
+.sp
+.SH WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the gather operation occurs in two phases.  The data is gathered from all the members of the first group, concatenated, and received by all the members of the second group.  Then the data is gathered from all the members of the second group, concatenated, and received by all the members of the first.  The send buffer arguments in the one group must be consistent with the receive buffer arguments in the other group, and vice versa.  The operation must exhibit symmetric, full-duplex behavior.
+.sp
+The first group defines the root process.  The root process uses MPI_ROOT as the value of \fIroot\fR.  All other processes in the first group use MPI_PROC_NULL as the value of \fIroot\fR.  All processes in the second group use the rank of the root process in the first group as the value of \fIroot\fR.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+.sp
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Gatherv
+.br
+MPI_Allgather

macx64/mpi/openmpi/share/man/man3/MPI_Alloc_mem.3

@@ -0,0 +1,87 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Alloc_mem 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Alloc_mem \fP \- Allocates a specified memory segment.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Alloc_mem(MPI_Aint \fIsize\fP, MPI_Info \fIinfo\fP, void *\fIbaseptr\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+size
+Size of memory segment in bytes (nonnegative integer).
+.ft R
+.TP 1i
+info
+Info argument (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+baseptr
+Pointer to beginning of memory segment allocated.
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Alloc_mem allocates \fIsize\fP bytes of memory. The starting address
+of this memory is returned in the variable \fIbase\fP.
+.sp
+
+.SH FORTRAN NOTES
+.ft R
+There is no portable FORTRAN 77 syntax for using MPI_Alloc_mem.
+There is no portable Fortran syntax for using pointers returned
+from MPI_Alloc_mem. However, MPI_Alloc_mem can be used with Sun
+Fortran compilers.
+.sp
+From FORTRAN 77, you can use the following non-standard
+declarations for the SIZE and BASEPTR arguments:
+.nf
+           INCLUDE "mpif.h"
+           INTEGER*MPI_ADDRESS_KIND SIZE, BASEPTR
+.fi
+.sp
+From either FORTRAN 77 or Fortran 90, you can use "Cray pointers"
+for the BASEPTR argument. Cray pointers are described further in
+the Fortran User's Guide and are supported by many Fortran compilers.
+For example,
+.sp
+.nf
+           INCLUDE "mpif.h"
+           REAL*4 A(100,100)
+           POINTER (BASEPTR, A)
+           INTEGER*MPI_ADDRESS_KIND SIZE
+
+           SIZE = 4 * 100 * 100
+           CALL MPI_ALLOC_MEM(SIZE,MPI_INFO_NULL,BASEPTR,IERR)
+
+           ! use A
+
+           CALL MPI_FREE_MEM(A, IERR)
+.fi
+.ft R
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Free_mem

macx64/mpi/openmpi/share/man/man3/MPI_Allreduce.3

@@ -0,0 +1,138 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2007-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Allreduce 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Allreduce, MPI_Iallreduce\fP \- Combines values from all processes and distributes the result back to all processes.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Allreduce(const void \fI*sendbuf\fP, void \fI*recvbuf\fP, int\fI count\fP,
+                  MPI_Datatype\fI datatype\fP, MPI_Op\fI op\fP, MPI_Comm\fI comm\fP)
+
+int MPI_Iallreduce(const void \fI*sendbuf\fP, void \fI*recvbuf\fP, int\fI count\fP,
+                   MPI_Datatype\fI datatype\fP, MPI_Op\fI op\fP, MPI_Comm\fI comm\fP,
+                   MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+sendbuf
+Starting address of send buffer (choice).
+.TP 1i
+count
+Number of elements in send buffer (integer).
+.TP 1i
+datatype
+Datatype of elements of send buffer (handle).
+.TP 1i
+op
+Operation (handle).
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+recvbuf
+Starting address of receive buffer (choice).
+.TP 1i
+request
+Request (handle, non-blocking only).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Same as MPI_Reduce except that the result appears in the receive buffer of all the group members.
+.sp
+\fBExample 1:\fR A routine that computes the product of a vector and an array that are distributed across a group of processes and returns the answer at all nodes (compare with Example 2, with MPI_Reduce, below).
+.sp
+.nf
+SUBROUTINE PAR_BLAS2(m, n, a, b, c, comm)
+REAL a(m), b(m,n)    ! local slice of array
+REAL c(n)            ! result
+REAL sum(n)
+INTEGER n, comm, i, j, ierr
+
+! local sum
+DO j= 1, n
+  sum(j) = 0.0
+  DO i = 1, m
+    sum(j) = sum(j) + a(i)*b(i,j)
+  END DO
+END DO
+
+! global sum
+CALL MPI_ALLREDUCE(sum, c, n, MPI_REAL, MPI_SUM, comm, ierr)
+
+! return result at all nodes
+RETURN
+.fi
+.sp
+\fBExample 2:\fR A routine that computes the product of a vector and an array that are distributed across a group of processes and returns the answer at node zero.
+.sp
+.nf
+SUBROUTINE PAR_BLAS2(m, n, a, b, c, comm)
+REAL a(m), b(m,n)    ! local slice of array
+REAL c(n)            ! result
+REAL sum(n)
+INTEGER n, comm, i, j, ierr
+
+! local sum
+DO j= 1, n
+  sum(j) = 0.0
+  DO i = 1, m
+    sum(j) = sum(j) + a(i)*b(i,j)
+  END DO
+END DO
+
+! global sum
+CALL MPI_REDUCE(sum, c, n, MPI_REAL, MPI_SUM, 0, comm, ierr)
+
+! return result at node zero (and garbage at the other nodes)
+RETURN
+.fi
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-reduce operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR at all processes.
+.sp
+Note that MPI_IN_PLACE is a special kind of value; it has the same restrictions on its use as MPI_BOTTOM.
+.sp
+Because the in-place option converts the receive buffer into a send-and-receive buffer, a Fortran binding that includes INTENT must mark these as INOUT, not OUT.
+.sp
+.SH WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+When the communicator is an inter-communicator, the reduce operation occurs in two phases.  The data is reduced from all the members of the first group and received by all the members of the second group.  Then the data is reduced from all the members of the second group and received by all the members of the first.  The operation exhibits a symmetric, full-duplex behavior.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+.SH NOTES ON COLLECTIVE OPERATIONS
+
+The reduction functions (
+.I MPI_Op
+) do not return an error value.  As a result,
+if the functions detect an error, all they can do is either call
+.I MPI_Abort
+or silently skip the problem.  Thus, if you change the error handler from
+.I MPI_ERRORS_ARE_FATAL
+to something else, for example,
+.I MPI_ERRORS_RETURN
+,
+then no error may be indicated.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Alltoall.3

@@ -0,0 +1,127 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Alltoall 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+\fBMPI_Alltoall, MPI_Ialltoall\fP \- All processes send data to all processes
+
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Alltoall(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP)
+
+int MPI_Ialltoall(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1.2i
+sendbuf
+Starting address of send buffer (choice).
+.TP 1.2i
+sendcount
+Number of elements to send to each process (integer).
+.TP 1.2i
+sendtype
+Datatype of send buffer elements (handle).
+.TP 1.2i
+recvcount
+Number of elements to receive from each process (integer).
+.TP 1.2i
+recvtype
+Datatype of receive buffer elements (handle).
+.TP 1.2i
+comm
+Communicator over which data is to be exchanged (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.2i
+recvbuf
+Starting address of receive buffer (choice).
+.TP 1.2i
+request
+Request (handle, non-blocking only).
+.ft R
+.TP 1.2i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Alltoall is a collective operation in which all processes send the same amount of data to each other, and receive the same amount of data from each other. The operation of this routine can be represented as follows, where each process performs 2n (n being the number of processes in communicator \fIcomm\fP) independent point-to-point communications (including communication with itself).
+.sp
+.nf
+	MPI_Comm_size(\fIcomm\fP, &n);
+	for (i = 0, i < n; i++)
+	    MPI_Send(\fIsendbuf\fP + i * \fIsendcount\fP * extent(\fIsendtype\fP),
+	        \fIsendcount\fP, \fIsendtype\fP, i, ..., \fIcomm\fP);
+	for (i = 0, i < n; i++)
+	    MPI_Recv(\fIrecvbuf\fP + i * \fIrecvcount\fP * extent(\fIrecvtype\fP),
+	        \fIrecvcount\fP, \fIrecvtype\fP, i, ..., \fIcomm\fP);
+.fi
+.sp
+Each process breaks up its local \fIsendbuf\fP into n blocks \- each
+containing \fIsendcount\fP elements of type \fIsendtype\fP \- and
+divides its \fIrecvbuf\fP similarly according to \fIrecvcount\fP and
+\fIrecvtype\fP. Process j sends the k-th block of its local
+\fIsendbuf\fP to process k, which places the data in the j-th block of
+its local \fIrecvbuf\fP. The amount of data sent must be equal to the
+amount of data received, pairwise, between every pair of processes.
+
+WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the gather operation occurs in two phases.  The data is gathered from all the members of the first group and received by all the members of the second group.  Then the data is gathered from all the members of the second group and received by all the members of the first.  The operation exhibits a symmetric, full-duplex behavior.
+.sp
+The first group defines the root process.  The root process uses MPI_ROOT as the value of \fIroot\fR.  All other processes in the first group use MPI_PROC_NULL as the value of \fIroot\fR.  All processes in the second group use the rank of the root process in the first group as the value of \fIroot\fR.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-to-all operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR.  In this case, \fIsendcount\fR and \fIsendtype\fR are ignored.  The input data of each process is assumed to be in the area where that process would receive its own contribution to the receive buffer.
+
+.sp
+.SH NOTES
+.sp
+All arguments on all processes are significant. The \fIcomm\fP argument,
+in particular, must describe the same communicator on all processes.
+.sp
+There are two MPI library functions that are more general than
+MPI_Alltoall. MPI_Alltoallv allows all-to-all communication to and
+from buffers that need not be contiguous; different processes may
+send and receive different amounts of data. MPI_Alltoallw expands
+MPI_Alltoallv's functionality to allow the exchange of data with
+different datatypes.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Alltoallv
+MPI_Alltoallw
+

macx64/mpi/openmpi/share/man/man3/MPI_Alltoallv.3

@@ -0,0 +1,154 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Alltoallv 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+\fBMPI_Alltoallv, MPI_Ialltoallv\fP \- All processes send different amount of data to, and receive different amount of data from, all processes
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Alltoallv(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], MPI_Datatype \fIsendtype\fP,
+	void *\fIrecvbuf\fP, const int\fI recvcounts\fP[],
+	const int \fIrdispls\fP[], MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP)
+
+int MPI_Ialltoallv(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], MPI_Datatype \fIsendtype\fP,
+	void *\fIrecvbuf\fP, const int\fI recvcounts\fP[],
+	const int \fIrdispls\fP[], MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP,
+	MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1.2i
+sendbuf
+Starting address of send buffer.
+.TP 1.2i
+sendcounts
+Integer array, where entry i specifies the number of elements to send
+to rank i.
+.TP 1.2i
+sdispls
+Integer array, where entry i specifies the displacement (offset from
+\fIsendbuf\fP, in units of \fIsendtype\fP) from which to send data to
+rank i.
+.TP 1.2i
+sendtype
+Datatype of send buffer elements.
+.TP 1.2i
+recvcounts
+Integer array, where entry j specifies the number of elements to
+receive from rank j.
+.TP 1.2i
+rdispls
+Integer array, where entry j specifies the displacement (offset from
+\fIrecvbuf\fP, in units of \fIrecvtype\fP) to which data from rank j
+should be written.
+.TP 1.2i
+recvtype
+Datatype of receive buffer elements.
+.TP 1.2i
+comm
+Communicator over which data is to be exchanged.
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.2i
+recvbuf
+Address of receive buffer.
+.TP 1.2i
+request
+Request (handle, non-blocking only).
+.ft R
+.TP 1.2i
+IERROR
+Fortran only: Error status.
+
+.SH DESCRIPTION
+.ft R
+MPI_Alltoallv is a generalized collective operation in which all
+processes send data to and receive data from all other processes. It
+adds flexibility to MPI_Alltoall by allowing the user to specify data
+to send and receive vector-style (via a displacement and element
+count). The operation of this routine can be thought of as follows,
+where each process performs 2n (n being the number of processes in
+communicator \fIcomm\fP) independent point-to-point communications
+(including communication with itself).
+.sp
+.nf
+	MPI_Comm_size(\fIcomm\fP, &n);
+	for (i = 0, i < n; i++)
+	    MPI_Send(\fIsendbuf\fP + \fIsdispls\fP[i] * extent(\fIsendtype\fP),
+	        \fIsendcounts\fP[i], \fIsendtype\fP, i, ..., \fIcomm\fP);
+	for (i = 0, i < n; i++)
+	    MPI_Recv(\fIrecvbuf\fP + \fIrdispls\fP[i] * extent(\fIrecvtype\fP),
+	        \fIrecvcounts\fP[i], \fIrecvtype\fP, i, ..., \fIcomm\fP);
+.fi
+.sp
+Process j sends the k-th block of its local \fIsendbuf\fP to process
+k, which places the data in the j-th block of its local
+\fIrecvbuf\fP.
+.sp
+When a pair of processes exchanges data, each may pass different
+element count and datatype arguments so long as the sender specifies
+the same amount of data to send (in bytes) as the receiver expects
+to receive.
+.sp
+Note that process i may send a different amount of data to process j
+than it receives from process j. Also, a process may send entirely
+different amounts of data to different processes in the communicator.
+
+.sp
+WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the gather operation occurs in two phases.  The data is gathered from all the members of the first group and received by all the members of the second group.  Then the data is gathered from all the members of the second group and received by all the members of the first.  The operation exhibits a symmetric, full-duplex behavior.
+.sp
+The first group defines the root process.  The root process uses MPI_ROOT as the value of \fIroot\fR.  All other processes in the first group use MPI_PROC_NULL as the value of \fIroot\fR.  All processes in the second group use the rank of the root process in the first group as the value of \fIroot\fR.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+.sp
+
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-to-all operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR.  In this case, \fIsendcounts\fR, \fIsdispls\fP, and \fIsendtype\fR are ignored.  The input data of each process is assumed to be in the area where that process would receive its own contribution to the receive buffer.
+
+.SH NOTES
+.sp
+The specification of counts and displacements should not cause
+any location to be written more than once.
+.sp
+All arguments on all processes are significant. The \fIcomm\fP argument,
+in particular, must describe the same communicator on all processes.
+.sp
+The offsets of \fIsdispls\fP and \fIrdispls\fP are measured in units
+of \fIsendtype\fP and \fIrecvtype\fP, respectively. Compare this to
+MPI_Alltoallw, where these offsets are measured in bytes.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Alltoall
+MPI_Alltoallw
+

macx64/mpi/openmpi/share/man/man3/MPI_Alltoallw.3

@@ -0,0 +1,156 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Alltoallw 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+\fBMPI_Alltoallw, MPI_Ialltoallw\fP \- All processes send data of different types to, and receive data of different types from, all processes
+
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Alltoallw(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], const MPI_Datatype \fIsendtypes\fP[],
+	void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[], const int \fIrdispls\fP[],
+	const MPI_Datatype \fIrecvtypes\fP[], MPI_Comm \fIcomm\fP)
+
+int MPI_Ialltoallw(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], const MPI_Datatype \fIsendtypes\fP[],
+	void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[], const int \fIrdispls\fP[],
+	const MPI_Datatype \fIrecvtypes\fP[], MPI_Comm \fIcomm\fP,
+	MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1.2i
+sendbuf
+Starting address of send buffer.
+.TP 1.2i
+sendcounts
+Integer array, where entry i specifies the number of elements to send
+to rank i.
+.TP 1.2i
+sdispls
+Integer array, where entry i specifies the displacement (in bytes,
+offset from \fIsendbuf\fP) from which to send data to rank i.
+.TP 1.2i
+sendtypes
+Datatype array, where entry i specifies the datatype to use when
+sending data to rank i.
+.TP 1.2i
+recvcounts
+Integer array, where entry j specifies the number of elements to
+receive from rank j.
+.TP 1.2i
+rdispls
+Integer array, where entry j specifies the displacement (in bytes,
+offset from \fIrecvbuf\fP) to which data from rank j should
+be written.
+.TP 1.2i
+recvtypes
+Datatype array, where entry j specifies the datatype to use when
+receiving data from rank j.
+.TP 1.2i
+comm
+Communicator over which data is to be exchanged.
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.2i
+recvbuf
+Address of receive buffer.
+.TP 1.2i
+request
+Request (handle, non-blocking only).
+.ft R
+.TP 1.2i
+IERROR
+Fortran only: Error status.
+
+.SH DESCRIPTION
+.ft R
+MPI_Alltoallw is a generalized collective operation in which all
+processes send data to and receive data from all other processes. It
+adds flexibility to MPI_Alltoallv by allowing the user to specify the
+datatype of individual data blocks (in addition to displacement and
+element count). Its operation can be thought of in the following way,
+where each process performs 2n (n being the number of processes in
+communicator \fIcomm\fP) independent point-to-point communications
+(including communication with itself).
+.sp
+.nf
+	MPI_Comm_size(\fIcomm\fP, &n);
+	for (i = 0, i < n; i++)
+	    MPI_Send(\fIsendbuf\fP + \fIsdispls\fP[i], \fIsendcounts\fP[i],
+	        \fIsendtypes\fP[i], i, ..., \fIcomm\fP);
+	for (i = 0, i < n; i++)
+	    MPI_Recv(\fIrecvbuf\fP + \fIrdispls\fP[i], \fIrecvcounts\fP[i],
+	        \fIrecvtypes\fP[i], i, ..., \fIcomm\fP);
+.fi
+.sp
+Process j sends the k-th block of its local \fIsendbuf\fP to process
+k, which places the data in the j-th block of its local
+\fIrecvbuf\fP.
+.sp
+When a pair of processes exchanges data, each may pass different
+element count and datatype arguments so long as the sender specifies
+the same amount of data to send (in bytes) as the receiver expects
+to receive.
+.sp
+Note that process i may send a different amount of data to process j
+than it receives from process j. Also, a process may send entirely
+different amounts and types of data to different processes in the
+communicator.
+
+WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the gather operation occurs in two phases.  The data is gathered from all the members of the first group and received by all the members of the second group.  Then the data is gathered from all the members of the second group and received by all the members of the first.  The operation exhibits a symmetric, full-duplex behavior.
+.sp
+The first group defines the root process.  The root process uses MPI_ROOT as the value of \fIroot\fR.  All other processes in the first group use MPI_PROC_NULL as the value of \fIroot\fR.  All processes in the second group use the rank of the root process in the first group as the value of \fIroot\fR.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+.sp
+
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-to-all operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR.  In this case, \fIsendcounts\fR, \fIsdispls\fP, and \fIsendtypes\fR are ignored.  The input data of each process is assumed to be in the area where that process would receive its own contribution to the receive buffer.
+
+.SH NOTES
+.sp
+The specification of counts, types, and displacements should not cause
+any location to be written more than once.
+.sp
+All arguments on all processes are significant. The \fIcomm\fP argument,
+in particular, must describe the same communicator on all processes.
+.sp
+The offsets of \fIsdispls\fP and \fIrdispls\fP are measured in bytes.
+Compare this to MPI_Alltoallv, where these offsets are measured in units
+of \fIsendtype\fP and \fIrecvtype\fP, respectively.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Alltoall
+MPI_Alltoallv
+

macx64/mpi/openmpi/share/man/man3/MPI_Attr_delete.3

@@ -0,0 +1,61 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Attr_delete 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Attr_delete\fP \- Deletes attribute value associated with a key -- use of this routine is deprecated.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Attr_delete(MPI_Comm\fI comm\fP, int\fI keyval\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator to which attribute is attached (handle).
+.TP 1i
+keyval
+The key value of the deleted attribute (integer).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+Note that use of this routine is \fIdeprecated\fP as of MPI-2, and
+was \fIdeleted\fP in MPI-3. Please use MPI_Comm_delete_attr.  This
+function does not have a C++ or mpi_f08 binding.
+.sp
+Delete attribute from cache by key. This function invokes the attribute delete function delete_fn specified when the keyval was created. The call will fail if the delete_fn function returns an error code other than MPI_SUCCESS.
+
+Whenever a communicator is replicated using the function MPI_Comm_dup, all callback copy functions for attributes that are currently set are invoked (in arbitrary order). Whenever a communicator is deleted using the function MPI_Comm_free, all callback delete functions for attributes that are currently set are invoked.
+
+
+.SH NOTES
+Note that it is not defined by the MPI standard what happens if the
+delete_fn callback invokes other MPI functions.  In Open MPI, it is
+not valid for delete_fn callbacks (or any of their children) to add or
+delete attributes on the same object on which the delete_fn callback
+is being invoked.
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+MPI_Comm_delete_attr
+.br
+

macx64/mpi/openmpi/share/man/man3/MPI_Attr_get.3

@@ -0,0 +1,61 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Attr_get 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Attr_get\fP \- Retrieves attribute value by key -- use of this routine is deprecated.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Attr_get(MPI_Comm \fIcomm\fP, int\fI keyval\fP,void\fI *attribute_val\fP,
+	int\fI *flag \fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator to which attribute is attached (handle).
+.TP 1i
+keyval
+ Key value (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+attribute_val
+Attribute value, unless flag = false.
+.TP 1i
+flag
+True if an attribute value was extracted; false if no attribute is associated with the key.
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Note that use of this routine is \fIdeprecated\fP as of MPI-2, and
+was \fIdeleted\fP in MPI-3. Please use MPI_Comm_get_attr.  This
+function does not have a C++ or mpi_f08 binding.
+.sp
+Retrieves attribute value by key. The call is erroneous if there is no key
+with value keyval. On the other hand, the call is correct if the key value exists, but no attribute is attached on comm for that key; in such case, the call returns flag = false. In particular MPI_KEYVAL_INVALID is an erroneous key value.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+
+.SH SEE ALSO
+
+MPI_Comm_get_attr
+.br

macx64/mpi/openmpi/share/man/man3/MPI_Attr_put.3

@@ -0,0 +1,64 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Attr_put 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Attr_put\fP \- Stores attribute value associated with a key -- use of this routine is deprecated.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Attr_put(MPI_Comm \fIcomm\fP, int\fI keyval\fP, void\fI *attribute_val\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator to which attribute will be attached (handle).
+.TP 1i
+keyval
+Key value, as returned by MPI_KEYVAL_CREATE (integer).
+.TP 1i
+attribute_val
+Attribute value.
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Note that use of this routine is \fIdeprecated\fP as of MPI-2, and
+was \fIdeleted\fP in MPI-3. Please use MPI_Comm_set_attr.  This
+function does not have a C++ or mpi_f08 binding.
+.sp
+MPI_Attr_put stores the stipulated attribute value attribute_val for subsequent retrieval by MPI_Attr_get. If the value is already present, then the outcome is as if MPI_Attr_delete was first called to delete the previous value (and the callback function delete_fn was executed), and a new value was next stored. The call is erroneous if there is no key with value keyval; in particular MPI_KEYVAL_INVALID is an erroneous key value. The call will fail if the delete_fn function returned an error code other than MPI_SUCCESS.
+
+.SH NOTES
+.ft R
+Values of the permanent attributes MPI_TAG_UB, MPI_HOST,
+MPI_IO, and MPI_WTIME_IS_GLOBAL may not be changed.
+.sp
+The type of the attribute value depends on whether C or Fortran is being used. In C, an attribute value is a pointer (void *); in Fortran, it is a single integer (not a pointer, since Fortran has no pointers and there are systems for which a pointer does not fit in an integer, e.g., any  32-bit address system that uses 64 bits for Fortran DOUBLE PRECISION).
+.sp
+If an attribute is already present, the delete function (specified when the corresponding keyval was created) will be called.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Comm_set_attr
+.br

macx64/mpi/openmpi/share/man/man3/MPI_Barrier.3

@@ -0,0 +1,52 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2014-2015 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Barrier 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Barrier, MPI_Ibarrier\fP \- Synchronization between MPI processes in a group
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Barrier(MPI_Comm \fIcomm\fP)
+
+int MPI_Ibarrier(MPI_Comm \fIcomm\fP, MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+request
+Request (handle, non-blocking only).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+An MPI barrier completes after all group members have entered the
+barrier.
+
+.SH WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the barrier operation is performed across all processes in both groups.  All processes in the first group may exit the barrier when all processes in the second group have entered the barrier.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+MPI_Bcast

macx64/mpi/openmpi/share/man/man3/MPI_Bcast.3

@@ -0,0 +1,81 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Bcast 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Bcast, MPI_Ibcast\fP \- Broadcasts a message from the process with rank \fIroot\fP to all other processes of the group.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Bcast(void \fI*buffer\fP, int\fI count\fP, MPI_Datatype\fI datatype\fP,
+	int\fI root\fP, MPI_Comm\fI comm\fP)
+
+int MPI_Ibcast(void \fI*buffer\fP, int\fI count\fP, MPI_Datatype\fI datatype\fP,
+	int\fI root\fP, MPI_Comm\fI comm\fP, MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT/OUTPUT PARAMETERS
+.ft R
+.TP 1i
+buffer
+Starting address of buffer (choice).
+.TP 1i
+count
+Number of entries in buffer (integer).
+.TP 1i
+datatype
+Data type of buffer (handle).
+.TP 1i
+root
+Rank of broadcast root (integer).
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+request
+Request (handle, non-blocking only).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Bcast broadcasts a message from the process with rank root to all processes of the group, itself included. It is called by all members of group using the same arguments for comm, root. On return, the contents of root's communication buffer has been copied to all processes.
+.sp
+General, derived datatypes are allowed for datatype. The type signature of count, datatype on any process must be equal to the type signature of count, datatype at the root. This implies that the amount of data sent must be equal to the amount received, pairwise between each process and the root. MPI_Bcast and all other data-movement collective routines make this restriction. Distinct type maps between sender and receiver are still allowed.
+.sp
+\fBExample:\fR Broadcast 100 ints from process 0 to every process in the group.
+.nf
+    MPI_Comm comm;
+    int array[100];
+    int root=0;
+    \&...
+    MPI_Bcast( array, 100, MPI_INT, root, comm);
+.fi
+.sp
+As in many of our sample code fragments, we assume that some of the variables (such as comm in the example above) have been assigned appropriate values.
+.sp
+.SH WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the root process in the first group broadcasts data to all the processes in the second group.  The first group defines the root process.  That process uses MPI_ROOT as the value of its \fIroot\fR argument.  The remaining processes use MPI_PROC_NULL as the value of their \fIroot\fR argument.  All processes in the second group use the rank of that root process in the first group as the value of their \fIroot\fR argument.   The receive buffer arguments of the processes in the second group must be consistent with the send buffer argument of the root process in the first group.
+.sp
+.SH NOTES
+This function does not support the in-place option.
+.sp
+
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Bsend.3

@@ -0,0 +1,96 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Bsend 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Bsend\fP \- Basic send with user-specified buffering.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Bsend(const void \fI*buf\fP, int\fI count\fP, MPI_Datatype\fI datatype\fP,
+	int\fI dest\fP, int\fI tag\fP, MPI_Comm\fI comm\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+buf
+Initial address of send buffer (choice).
+.TP 1i
+count
+Number of entries in send buffer (nonnegative integer).
+.TP 1i
+datatype
+Datatype of each send buffer element (handle).
+.TP 1i
+dest
+Rank of destination (integer).
+.TP 1i
+tag
+Message tag (integer).
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Bsend performs a buffered-mode, blocking send.
+
+.SH NOTES
+.ft R
+This send is provided as a convenience function; it allows the user to send messages without worrying about where they are buffered (because the user must have provided buffer space with MPI_Buffer_attach).
+.sp
+In deciding how much buffer space to allocate, remember that the buffer space
+is not available for reuse by subsequent \fIMPI_Bsend\fPs unless you are certain
+that the message
+has been received (not just that it should have been received).  For example,
+this code does not allocate enough buffer space:
+.nf
+
+    MPI_Buffer_attach( b, n*sizeof(double) + MPI_BSEND_OVERHEAD );
+    for (i=0; i<m; i++) {
+        MPI_Bsend( buf, n, MPI_DOUBLE, ... );
+    }
+
+.fi
+because only enough buffer space is provided for a single send, and the
+loop may start a second
+.I MPI_Bsend
+before the first is done making use of the
+buffer.
+
+In C, you can
+force the messages to be delivered by
+MPI_Buffer_detach( &b, &n );
+MPI_Buffer_attach( b, n );
+(The
+.I MPI_Buffer_detach
+will not complete until all buffered messages are
+delivered.)
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Buffer_attach
+MPI_Ibsend
+MPI_Bsend_init
+

macx64/mpi/openmpi/share/man/man3/MPI_Bsend_init.3

@@ -0,0 +1,72 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Bsend_init 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Bsend_init\fP \- Builds a handle for a buffered send.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Bsend_init(const void \fI*buf\fP, int\fI count\fP, MPI_Datatype\fI datatype\fP,
+	int\fI dest\fP, int\fI tag\fP, MPI_Comm\fI comm\fP, MPI_Request\fI *request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+buf
+Initial address of send buffer (choice).
+.TP 1i
+count
+Number of elements sent (integer).
+.TP 1i
+datatype
+Type of each element (handle).
+.TP 1i
+dest
+Rank of destination (integer).
+.TP 1i
+tag
+Message tag (integer).
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+request
+Communication request (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Creates a persistent communication request for a buffered mode send, and binds to it all the arguments of a send operation.
+.sp
+A communication (send or receive) that uses a persistent request is initiated by the function MPI_Start.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Send_init
+MPI_Ssend_init
+MPI_Rsend_init
+MPI_Recv_init
+MPI_Start
+MPI_Startall
+

macx64/mpi/openmpi/share/man/man3/MPI_Buffer_attach.3

@@ -0,0 +1,68 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Buffer_attach 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Buffer_attach\fP \- Attaches a user-defined buffer for sending.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Buffer_attach(void \fI*buf\fP, int\fI size\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+buf
+Initial buffer address (choice).
+.TP 1i
+size
+Buffer size, in bytes (integer).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Provides to MPI a buffer in the user's memory to be used for buffering outgoing messages. The buffer is used only by messages sent in buffered mode. Only one buffer can be attached to a process at a time.
+
+.SH NOTES
+.ft R
+The size given should be the sum of the sizes of all outstanding Bsends that you intend to have, plus MPI_BSEND_OVERHEAD bytes for each Bsend that you do. For the purposes of calculating size, you should use MPI_Pack_size. In other words, in the code
+.sp
+.nf
+    MPI_Buffer_attach( buf, size );
+    MPI_Bsend( \&..., count=20, datatype=type1, \&... );
+    \&...
+    MPI_Bsend( \&..., count=40, datatype=type2, \&... );
+.fi
+.sp
+the value of size in the MPI_Buffer_attach call should be greater than the value computed by
+.sp
+.nf
+    MPI_Pack_size( 20, type1, comm, &s1 );
+    MPI_Pack_size( 40, type2, comm, &s2 );
+    size = s1 + s2 + 2 * MPI_BSEND_OVERHEAD;
+.fi
+.sp
+MPI_BSEND_OVERHEAD gives the maximum amount of buffer space that may be used by the Bsend routines. This value is in mpi.h for C and mpif.h for Fortran.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Buffer_detach
+

macx64/mpi/openmpi/share/man/man3/MPI_Buffer_detach.3

@@ -0,0 +1,85 @@
+.\" -*- nroff -*-
+.\" Copyright 2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Buffer_detach 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Buffer_detach\fP \- Removes an existing buffer (for use in MPI_Bsend, etc.)
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Buffer_detach(void \fI*buf\fP, int\fI *size\fP)
+
+.fi
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+buf
+Initial buffer address (choice).
+.TP 1i
+size
+Buffer size, in bytes (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Detach the buffer currently associated with MPI. The call returns the address and the size of the detached buffer. This operation will block until all messages currently in the buffer have been transmitted. Upon return of this function, the user may reuse or deallocate the space taken by the buffer.
+.sp
+\fBExample:\fP Calls to attach and detach buffers.
+.sp
+.nf
+    #define BUFFSIZE 10000
+    int size
+    char *buff;
+    MPI_Buffer_attach( malloc(BUFFSIZE), BUFFSIZE);
+    /* a buffer of 10000 bytes can now be used by MPI_Bsend */
+    MPI_Buffer_detach( &buff, &size);
+    /* Buffer size reduced to zero */
+    MPI_Buffer_attach( buff, size);
+    /* Buffer of 10000 bytes available again */
+.fi
+
+.SH NOTES
+.ft R
+The reason that MPI_Buffer_detach returns the address and size of the buffer being detached is to allow nested libraries to replace and restore the buffer. For example, consider
+.sp
+.nf
+    int size, mysize, idummy;
+    void *ptr, *myptr, *dummy;
+    MPI_Buffer_detach( &ptr, &size );
+    MPI_Buffer_attach( myptr, mysize );
+    \&...
+    \&... library code \&...
+    \&...
+    MPI_Buffer_detach( &dummy, &idummy );
+    MPI_Buffer_attach( ptr, size );
+.fi
+.sp
+This is much like the action of the UNIX signal routine and has the same strengths (it's simple) and weaknesses (it only works for nested usages).
+.sp
+\fBFor Fortran:\fP The Fortran binding for this routine is different. Because Fortran does not have pointers, it is impossible to provide a way to use the output of this routine to exchange buffers. In this case, only the size field is set.
+.sp
+\fBFor C:\fP Even though the buf argument is declared as void, it is really the address of a void pointer. See Rationale, below, for more details.
+.sp
+Even though the C functions MPI_Buffer_attach and
+MPI_Buffer_detach both have a first argument of type void*, these arguments are used differently: A pointer to the buffer is passed to MPI_Buffer_attach; the address of the pointer is passed to MPI_Buffer_detach, so that this call can return the pointer value.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Buffer_attach
+MPI_Bsend

macx64/mpi/openmpi/share/man/man3/MPI_Cancel.3

@@ -0,0 +1,67 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cancel 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cancel\fP \- Cancels a communication request.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cancel(MPI_Request\fI *request\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+request
+Communication request (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+The MPI_Cancel operation allows pending communications to be canceled. This is required for cleanup. Posting a send or a receive ties up user resources (send or receive buffers), and a cancel may be needed to free these resources gracefully.
+.sp
+A call to MPI_Cancel marks for cancellation a pending, nonblocking communication operation (send or receive). The cancel call is local. It returns immediately, possibly before the communication is actually canceled. It is still necessary to complete a communication that has been marked for cancellation, using a call to MPI_Request_free, MPI_Wait, or MPI_Test (or any of the derived operations).
+.sp
+If a communication is marked for cancellation, then an MPI_Wait call for that communication is guaranteed to return, irrespective of the activities of other processes (i.e., MPI_Wait behaves as a local function); similarly if MPI_Test is repeatedly called in a busy wait loop for a canceled communication, then MPI_Test will eventually be successful.
+.sp
+MPI_Cancel can be used to cancel a communication that uses a persistent request (see Section 3.9 in the MPI-1 Standard, "Persistent Communication Requests") in the same way it is used for nonpersistent requests. A successful cancellation cancels the active communication, but not the request itself. After the call to MPI_Cancel and the subsequent call to MPI_Wait or MPI_Test, the request becomes inactive and can be activated for a new communication.
+.sp
+The successful cancellation of a buffered send frees the buffer space occupied by the pending message.
+.sp
+Either the cancellation succeeds or the communication succeeds, but not both. If a send is marked for cancellation, then it must be the case that either the send completes normally, in which case the message sent is received at the destination process, or that the send is successfully canceled, in which case no part of the message is received at the destination. Then, any matching receive has to be satisfied by another send. If a receive is marked for cancellation, then it must be the case that either the receive completes normally, or that the receive is successfully canceled, in which case no part of the receive buffer is altered. Then, any matching send has to be satisfied by another receive.
+.sp
+If the operation has been canceled, then information to that effect will be returned in the status argument of the operation that completes the communication.
+
+.SH NOTES
+.ft R
+The primary expected use of MPI_Cancel is in multi-buffering schemes,
+where speculative MPI_Irecvs are made.  When the
+computation completes, some of these requests may remain;
+using MPI_Cancel allows the user to cancel these unsatisfied requests.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Probe
+MPI_Iprobe
+MPI_Test_cancelled
+MPI_Cart_coords
+

macx64/mpi/openmpi/share/man/man3/MPI_Cart_coords.3

@@ -0,0 +1,50 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_coords 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_coords\fP \- Determines process coords in Cartesian topology given rank in group.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_coords(MPI_Comm \fIcomm\fP, int\fI rank\fP, int\fI maxdims\fP,
+	int\fI coords\fP[])
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+.TP 1i
+rank
+Rank of a process within group of comm (integer).
+.TP 1i
+maxdims
+ Length of vector coords in the calling program (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+coords
+Integer array (of size ndims,which was defined by MPI_Cart_create call) containing the Cartesian coordinates of specified process (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Cart_coords provies a mapping of ranks to Cartesian coordinates.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Cart_create.3

@@ -0,0 +1,59 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_create 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_create\fP \- Makes a new communicator to which Cartesian topology information has been attached.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_create(MPI_Comm\fI comm_old\fP, int\fI ndims\fP, const int\fI dims\fP[],
+	const int\fI periods\fP[], int\fI reorder\fP, MPI_Comm\fI *comm_cart\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm_old
+Input communicator (handle).
+.TP 1i
+ndims
+Number of dimensions of Cartesian grid (integer).
+.TP 1i
+dims
+Integer array of size ndims specifying the number of processes in each
+dimension.
+.TP 1i
+periods
+Logical array of size ndims specifying whether the grid is periodic (true)
+or not (false) in each dimension.
+.TP 1i
+reorder
+Ranking may be reordered (true) or not (false) (logical).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+comm_cart
+Communicator with new Cartesian topology (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Cart_create returns a handle to a new communicator to which the Cartesian topology information is attached. If reorder = false then the rank of each process in the new group is identical to its rank in the old group. Otherwise, the function may reorder the processes (possibly so as to choose a good embedding of the virtual topology onto the physical machine). If the total size of the Cartesian grid is smaller than the size of the group of comm, then some processes are returned MPI_COMM_NULL, in analogy to MPI_Comm_split. The call is erroneous if it specifies a grid that is larger than the group size.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Cart_get.3

@@ -0,0 +1,59 @@
+.\" -*- nroff -*-
+.\" Copyright 2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_get 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_get\fP \-  Retrieves Cartesian topology information associated with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_get(MPI_Comm\fI comm\fP, int\fI maxdims\fP, int\fI dims\fP[], int\fI periods\fP[],
+	int\fI coords\fP[])
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+.TP 1i
+maxdims
+Length of vectors dims, periods, and coords in the calling program (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+dims
+Number of processes for each Cartesian dimension (array of integers).
+.TP 1i
+periods
+Periodicity (true/false) for each Cartesian dimension (array of logicals).
+.TP 1i
+coords
+Coordinates of calling process in Cartesian structure (array of integers).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+The functions MPI_Cartdim_get and MPI_Cart_get return the Cartesian topology information that was associated with a communicator by MPI_Cart_create.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Cartdim_get
+MPI_Cart_create

macx64/mpi/openmpi/share/man/man3/MPI_Cart_map.3

@@ -0,0 +1,61 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_map 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_map \fP \-  Maps process to Cartesian topology information.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_map(MPI_Comm \fIcomm\fP, int\fI ndims\fP, const int\fI dims\fP[],
+	const int\fI periods\fP[], int\fI *newrank\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Input communicator (handle).
+.TP 1i
+ndims
+Number of dimensions of Cartesian structure (integer).
+.TP 1i
+dims
+Integer array of size ndims specifying the number of processes in each
+coordinate direction.
+.TP 1i
+periods
+Logical array of size ndims specifying the periodicity specification in each coordinate direction.
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newrank
+Reordered rank of the calling process; MPI_UNDEFINED if calling process does not belong to grid (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Cart_map and MPI_Graph_map can be used to implement all other topology functions. In general they will not be called by the user directly, unless he or she is creating additional virtual topology capability other than that provided by MPI.
+.sp
+MPI_Cart_map computes an "optimal" placement for the calling process on the physical machine. A possible implementation of this function is to always return the rank of the calling process, that is, not to perform any reordering.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Graph_map

macx64/mpi/openmpi/share/man/man3/MPI_Cart_rank.3

@@ -0,0 +1,50 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_rank 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_rank \fP \-  Determines process rank in communicator given Cartesian location.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_rank(MPI_Comm \fIcomm\fP, int\fI coords\fP[], int\fI *rank\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+.TP 1i
+coords
+Integer array (of size ndims, which was defined by MPI_Cart_create call) specifying the Cartesian coordinates of a process.
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+rank
+Rank of specified process (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+For a process group with Cartesian structure, the function MPI_Cart_rank
+translates the logical process coordinates to process ranks as they are used by the point-to-point routines.  For dimension i with periods(i) = true, if the coordinate, coords(i), is out of range, that is, coords(i) < 0 or  coords(i) >= dims(i), it is shifted back to the interval  0 =< coords(i) < dims(i) automatically. Out-of-range coordinates are erroneous for nonperiodic dimensions.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.sp
+MPI_Cart_create

macx64/mpi/openmpi/share/man/man3/MPI_Cart_shift.3

@@ -0,0 +1,80 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Cart_shift 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_shift \fP \-  Returns the shifted source and destination ranks, given a shift direction and amount.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_shift(MPI_Comm \fIcomm\fP, int\fI direction\fP, int\fI disp\fP,
+	int\fI *rank_source\fP, int\fI *rank_dest\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+.TP 1i
+direction
+Coordinate dimension of shift (integer).
+.TP 1i
+disp
+Displacement ( > 0: upward shift, < 0: downward shift) (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+rank_source
+Rank of source process (integer).
+.TP 1i
+rank_dest
+Rank of destination process (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+If the process topology is a Cartesian structure, an MPI_Sendrecv operation is likely to be used along a coordinate direction to perform a shift of data. As input, MPI_Sendrecv takes the rank of a source process for the receive, and the rank of a destination process for the send. If the function MPI_Cart_shift is called for a Cartesian process group, it provides the calling process with the above identifiers, which then can be passed to MPI_Sendrecv. The user specifies the coordinate direction and the size of the step (positive or negative). The function is local.
+.sp
+The direction argument indicates the dimension of the shift, i.e., the coordinate whose value is modified by the shift. The coordinates are numbered from 0 to ndims-1, where ndims is the number of dimensions.
+.sp
+\fBNote:\fP  The direction argument is in the range [0, n-1] for an n-dimensional Cartesian mesh.
+.sp
+Depending on the periodicity of the Cartesian group in the specified coordinate direction, MPI_Cart_shift provides the identifiers for a circular or an end-off shift. In the case of an end-off shift, the value MPI_PROC_NULL may be returned in rank_source or rank_dest, indicating that the source or the destination for the shift is out of range.
+.sp
+\fBExample:\fP The communicator, comm, has a two-dimensional, periodic, Cartesian  topology associated with it. A two-dimensional array of REALs is stored one element per process, in variable A. One wishes to skew this array, by shifting column i (vertically, i.e., along the column) by i steps.
+.sp
+.nf
+  \&....
+  C find process rank
+        CALL MPI_COMM_RANK(comm, rank, ierr)
+  C find Cartesian coordinates
+        CALL MPI_CART_COORDS(comm, rank, maxdims, coords,
+                             ierr)
+  C compute shift source and destination
+        CALL MPI_CART_SHIFT(comm, 0, coords(2), source,
+                            dest, ierr)
+  C skew array
+        CALL MPI_SENDRECV_REPLACE(A, 1, MPI_REAL, dest, 0,
+                                  source, 0, comm, status,
+                                  ierr)
+.fi
+
+.SH NOTE
+In Fortran, the dimension indicated by DIRECTION = i has DIMS(i+1) nodes, where DIMS is the array that was used to create the grid. In C, the dimension indicated by direction = i is the dimension specified by dims[i].
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Cart_sub.3

@@ -0,0 +1,62 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_sub 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_sub \fP \- Partitions a communicator into subgroups, which form lower-dimensional Cartesian subgrids.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_sub(MPI_Comm \fIcomm\fP, const int\fI remain_dims\fP[], MPI_Comm\fI *comm_new\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+.TP 1i
+remain_dims
+The ith entry of remain_dims specifies whether the ith dimension is kept in the subgrid (true) or is dropped (false) (logical vector).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+comm_new
+Communicator containing the subgrid that includes the calling process (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+If a Cartesian topology has been created with MPI_Cart_create, the function  MPI_Cart_sub can be used to partition the communicator group into subgroups that form lower-dimensional Cartesian subgrids, and to build for each subgroup a communicator with the associated subgrid Cartesian topology. (This function is closely related to MPI_Comm_split.)
+.sp
+\fBExample:\fP Assume that  MPI_Cart_create( \&..., comm) has defined a (2 x 3 x 4)   grid. Let  remain_dims = (true, false, true). Then a call to
+.sp
+.nf
+    MPI_Cart_sub(comm, remain_dims, comm_new)
+.fi
+.sp
+will create three communicators, each with eight processes in a 2 x 4 Cartesian topology. If remain_dims = (false, false, true) then the call to MPI_Cart_sub(comm, remain_dims, comm_new) will create six nonoverlapping communicators, each with four processes, in a one-dimensional Cartesian topology.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Cart_create
+MPI_Comm_split
+

macx64/mpi/openmpi/share/man/man3/MPI_Cartdim_get.3

@@ -0,0 +1,50 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cartdim_get 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cartdim_get \fP \-  Retrieves Cartesian topology information associated with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cartdim_get(MPI_Comm\fI comm\fP, int\fI *ndims\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+ndims
+Number of dimensions of the Cartesian structure (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Cartdim_get returns the number of dimensions of the Cartesian structure.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Cart_get
+MPI_Cart_create
+

macx64/mpi/openmpi/share/man/man3/MPI_Close_port.3

@@ -0,0 +1,40 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Close_port 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Close_port \fP \- Releases the specified network address.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Close_port(const char *\fIport_name\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+port_name
+A port (string).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Close_port releases the network address represented by \fIport_name\fP.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_accept.3

@@ -0,0 +1,62 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2009-2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2007, Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_accept 3OpenMPI "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_accept \fP \- Establishes communication with a client.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_accept(const char *\fIport_name\fP, MPI_Info \fIinfo\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP, MPI_Comm *\fInewcomm\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+port_name
+Port name (string, used only on \fIroot\fP).
+.TP 1i
+info
+Options given by root for the accept (handle, used only on root). No options currently supported.
+.TP 1i
+root
+Rank in \fIcomm\fP of root node (integer).
+.TP 1i
+comm
+Intracommunicator over which call is collective (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+Intercommunicator with client as remote group (handle)
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_accept establishes communication with a client. It is collective over the calling communicator. It returns an intercommunicator that allows communication with the client, after the client has connected with the MPI_Comm_accept function using the MPI_Comm_connect function.
+.sp
+The \fIport_name\fP must have been established through a call to MPI_Open_port on the root.
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+.SH SEE ALSO
+MPI_Comm_connect
+MPI_Open_port
+.br
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_c2f.3

@@ -0,0 +1 @@
+.so man3/MPI_Comm_f2c.3

macx64/mpi/openmpi/share/man/man3/MPI_Comm_call_errhandler.3

@@ -0,0 +1,66 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_call_errhandler 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+
+\fBMPI_Comm_call_errhandler\fP \- Passes the supplied error code to the
+error handler assigned to a communicator
+
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_call_errhandler(MPI_Comm \fIcomm\fP, int \fIerrorcode\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1.4i
+comm
+communicator with error handler (handle).
+.ft R
+.TP 1.4i
+errorcode
+error code (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.4i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function invokes the error handler assigned to the communicator
+\fIcomm\fP with the supplied error code \fIerrorcode\fP. If the error
+handler was successfully called, the process is not aborted, and the
+error handler returns, this function returns MPI_SUCCESS.
+
+.SH NOTES
+.ft R
+Users should note that the default error handler is
+MPI_ERRORS_ARE_FATAL. Thus, calling this function will abort the
+processes in \fIcomm\fP if the default error handler has not been
+changed.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Comm_create_errhandler
+MPI_Comm_set_errhandler
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_compare.3

@@ -0,0 +1,47 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_compare 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_compare \fP \-  Compares two communicators.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_compare(MPI_Comm \fIcomm1\fP, MPI_Comm\fI comm2\fP, int\fI *result\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm1
+Comm1 (handle).
+.TP 1i
+comm2
+Comm2 (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+result
+Result of comparison (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_IDENT results if and only if comm1 and comm2 are handles for the same object (identical groups and same contexts). MPI_CONGRUENT results if the underlying groups are identical in constituents and rank order; these communicators differ only by context. MPI_SIMILAR results of the group members of both communicators are the same but the rank order differs. MPI_UNEQUAL results otherwise.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_connect.3

@@ -0,0 +1,67 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2007-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_connect 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_connect \fP \- Establishes communication with a server.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_connect(const char *\fIport_name\fP, MPI_Info \fIinfo\fP, int \fIroot\fP,
+	MPI_Comm \fIcomm\fP, MPI_Comm *\fInewcomm\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+port_name
+Port name (string, used only on \fIroot\fP).
+.TP 1i
+info
+Options given by root for the connect (handle, used only on root). No options currently supported.
+.TP 1i
+root
+Rank in \fIcomm\fP of root node (integer).
+.TP 1i
+comm
+Intracommunicator over which call is collective (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+Intercommunicator with client as remote group (handle)
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_connect establishes communication with a server specified by \fIport_name\fP. It is collective over the calling communicator and returns an intercommunicator in which the remote group participated in an MPI_Comm_accept. The MPI_Comm_connect call must only be called after the MPI_Comm_accept call has been made by the MPI job acting as the server.
+.sp
+If the named port does not exist (or has been closed), MPI_Comm_connect raises an error of class MPI_ERR_PORT.
+.sp
+MPI provides no guarantee of fairness in servicing connection attempts. That is, connection attempts are not necessarily satisfied in the order in which they were initiated, and competition from other connection attempts may prevent a particular connection attempt from being satisfied.
+
+The \fIport_name\fP parameter is the address of the server. It must be the same as the name returned by MPI_Open_port on the server.
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+.SH SEE ALSO
+MPI_Comm_accept
+MPI_Open_port
+.br
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_create.3

@@ -0,0 +1,71 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_create 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_create\fP \- Creates a new communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_create(MPI_Comm \fIcomm\fP, MPI_Group\fI group\fP, MPI_Comm\fI *newcomm\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+.TP 1i
+group
+Group, which is a subset of the group of comm (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+New communicator (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function creates a new communicator newcomm with communication
+group defined by group and a new context. The function sets
+\fInewcomm\fR to a new communicator that spans all the processes that
+are in the group.  It sets \fInewcomm\fR to MPI_COMM_NULL for
+processes that are not in the group.
+
+Each process must call with a \fIgroup\fR argument that is a subgroup
+of the group associated with \fIcomm\fR; this could be
+MPI_GROUP_EMPTY. The processes may specify different values for the
+\fIgroup\fR argument. If a process calls with a non-empty \fIgroup\fR,
+then all processes in that group must call the function with the same
+\fIgroup\fR as argument, that is: the same processes in the same
+order. Otherwise the call is erroneous.
+.sp
+.LP
+
+.SH NOTES
+MPI_Comm_create provides a means of making a subset of processes for the purpose of separate MIMD computation, with separate communication space. \fInewcomm\fR, which is created by MPI_Comm_create, can be used in subsequent calls to MPI_Comm_create (or other communicator constructors) to further subdivide a computation into parallel sub-computations. A more general service is provided by MPI_Comm_split.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Comm_split
+.sp
+MPI_Intercomm_create
+MPI_Comm_create_group

macx64/mpi/openmpi/share/man/man3/MPI_Comm_create_errhandler.3

@@ -0,0 +1,76 @@
+.\" -*- nroff -*-
+.\" Copyright 2009-2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_create_errhandler 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_create_errhandler \fP \- Creates an error handler that can be attached to communicators.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_create_errhandler(MPI_Comm_errhandler_function \fI*function\fP,
+	MPI_Errhandler *\fIerrhandler\fP)
+
+.fi
+.SH DEPRECATED TYPE NAME NOTE
+.ft R
+MPI-2.2 deprecated the MPI_Comm_errhandler_fn and
+MPI::Comm::Errhandler_fn types in favor of
+MPI_Comm_errhandler_function and MPI::Comm::Errhandler_function,
+respectively.  Open MPI supports both names (indeed, the _fn names are
+typedefs to the _function names).
+
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+function
+User-defined error handling procedure (function).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+errhandler
+MPI error handler (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_create_errhandler creates an error handler that can be attached to communicators. This function is identical to MPI_Errhandler_create, the use of which is deprecated.
+.sp
+In C, the user routine should be a function of type MPI_Comm_errhandler_function, which is defined as
+.sp
+.nf
+    typedef void MPI_Comm_errhandler_function(MPI_Comm *, int *, \&...);
+.fi
+.sp
+The first argument is the communicator in use. The second is the error code
+to be returned by the MPI routine that raised the error. This typedef replaces MPI_Handler_function, the use of which is deprecated.
+.sp
+In Fortran, the user routine should be of this form:
+.sp
+.nf
+    SUBROUTINE COMM_ERRHANDLER_FUNCTION(COMM, ERROR_CODE, \&...)
+        INTEGER COMM, ERROR_CODE
+.fi
+.sp
+In C++, the user routine should be of this form:
+.sp
+.nf
+    typedef void MPI::Comm::Errhandler_function(MPI_Comm &, int *, \&...);
+.fi
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_create_group.3

@@ -0,0 +1,75 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_create_group 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_create_group\fP \- Creates a new communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_create_group(MPI_Comm \fIcomm\fP, MPI_Group\fI group\fP, int\fI tag\fP, MPI_Comm\fI *newcomm\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+.TP 1i
+group
+Group, which is a subset of the group of comm (handle).
+.TP 1i
+tag
+Tag (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+New communicator (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_create_group is similar to MPI_Comm_create; however,
+MPI_Comm_create must be called by all processes in the group of
+comm, whereas MPI_Comm_create_group must be called by all processes in group,
+which is a subgroup of the group of \fIcomm\fP. In addition, MPI_Comm_create_group
+requires that \fIcomm\fP is an intracommunicator. MPI_Comm_create_group returns a new
+intracommunicator, \fInewcomm\fP, for which the group argument defines the communication
+group. No cached information propagates from \fIcomm\fP to \fInewcomm\fP.
+
+Each process must provide a group argument that is a subgroup of the group associated with \fIcomm\fP;
+this could be MPI_GROUP_EMPTY. If a non-empty group is specified, then all processes in that
+group must call the function, and each of these processes must provide the same arguments,
+including a group that contains the same members with the same ordering. Otherwise
+the call is erroneous. If the calling process is a member of the group given as the \fIgroup\fP
+argument, then newcomm is a communicator with group as its associated group. If the
+calling process is not a member of group, e.g., \fIgroup\fP is MPI_GROUP_EMPTY, then the call
+is a local operation and MPI_COMM_NULL is returned as \fInewcomm\fP.
+
+.sp
+.LP
+
+.SH NOTES
+MPI_Comm_create_group provides a means of making a subset of processes for the purpose of separate MIMD computation, with separate communication space. \fInewcomm\fR, which is created by MPI_Comm_create_group, can be used in subsequent calls to MPI_Comm_create_group (or other communicator constructors) to further subdivide a computation into parallel sub-computations. A more general service is provided by MPI_Comm_split.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+MPI_Comm_create
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_create_keyval.3

@@ -0,0 +1,119 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_create_keyval 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_create_keyval\fP \- Generates a new attribute key.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_create_keyval(MPI_Comm_copy_attr_function
+	*\fIcomm_copy_attr_fn\fP, MPI_Comm_delete_attr_function
+	*\fIcomm_delete_attr_fn\fP, int *\fIcomm_keyval\fP,
+	void *\fIextra_state\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm_copy_attr_fn
+Copy callback function for \fIcomm_keyval\fP (function).
+.TP 1i
+comm_delete_attr_fn
+Delete callback function for \fIcomm_keyval\fP (function).
+.TP 1i
+extra_state
+Extra state for callback functions.
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm_keyval
+Key value for future access (integer).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function replaces MPI_Keyval_create, the use of which is deprecated. The C binding is identical. The Fortran binding differs in that \fIextra_state\fP is an address-sized integer. Also, the copy and delete callback functions have Fortran bindings that are consistent with address-sized attributes.
+.sp
+The argument \fIcomm_copy_attr_fn\fP may be specified as MPI_COMM_NULL_COPY_FN or MPI_COMM_DUP_FN from C, C++, or Fortran. MPI_COMM_NULL_COPY_FN is a function that does nothing more than returning \fIflag\fP = 0 and MPI_SUCCESS. MPI_COMM_DUP_FN is a simple-minded copy function that sets \fIflag\fP = 1, returns the value of \fIattribute_val_in\fP in \fIattribute_val_out\fP, and returns MPI_SUCCESS. These replace the MPI-1 predefined callbacks MPI_NULL_COPY_FN and MPI_DUP_FN, the use of which is deprecated.
+.sp
+The C callback functions are:
+.sp
+.nf
+typedef int MPI_Comm_copy_attr_function(MPI_Comm \fIoldcomm\fP, int \fIcomm_keyval\fP,
+             void *\fIextra_state\fP, void *\fIattribute_val_in\fP,
+             void *\fIattribute_val_out\fP, int *\fIflag\fP);
+.fi
+and
+.nf
+typedef int MPI_Comm_delete_attr_function(MPI_Comm \fIcomm\fP, int \fIcomm_keyval\fP,
+             void *\fIattribute_val\fP, void *\fIextra_state\fP);
+.fi
+.sp
+which are the same as the MPI-1.1 calls but with a new name. The old names are deprecated.
+.sp
+The Fortran callback functions are:
+.sp
+.nf
+SUBROUTINE COMM_COPY_ATTR_FN(\fIOLDCOMM, COMM_KEYVAL, EXTRA_STATE,
+             ATTRIBUTE_VAL_IN, ATTRIBUTE_VAL_OUT, FLAG, IERROR\fP)
+    INTEGER \fIOLDCOMM, COMM_KEYVAL, IERROR\fP
+    INTEGER(KIND=MPI_ADDRESS_KIND) \fIEXTRA_STATE, ATTRIBUTE_VAL_IN,
+        ATTRIBUTE_VAL_OUT\fP
+    LOGICAL \fIFLAG\fP
+.fi
+and
+.nf
+SUBROUTINE COMM_DELETE_ATTR_FN(\fICOMM, COMM_KEYVAL, ATTRIBUTE_VAL, EXTRA_STATE,
+             IERROR\fP)
+    INTEGER \fICOMM, COMM_KEYVAL, IERROR\fP
+    INTEGER(KIND=MPI_ADDRESS_KIND) \fIATTRIBUTE_VAL, EXTRA_STATE\fP
+.fi
+.sp
+The C++ callbacks are:
+.sp
+.nf
+typedef int MPI::Comm::Copy_attr_function(const MPI::Comm& \fIoldcomm\fP,
+             int \fIcomm_keyval\fP, void* \fIextra_state\fP, void* \fIattribute_val_in\fP,
+             void* \fIattribute_val_out\fP, bool& \fIflag\fP);
+.fi
+and
+.nf
+typedef int MPI::Comm::Delete_attr_function(MPI::Comm& \fIcomm\fP,
+             int \fIcomm_keyval\fP, void* \fIattribute_val\fP, void* \fIextra_state\fP);
+.fi
+
+.SH FORTRAN 77 NOTES
+.ft R
+The MPI standard prescribes portable Fortran syntax for
+the \fIEXTRA_STATE\fP argument only for Fortran 90. FORTRAN 77
+users may use the non-portable syntax
+.sp
+.nf
+     INTEGER*MPI_ADDRESS_KIND \fIEXTRA_STATE\fP
+.fi
+.sp
+where MPI_ADDRESS_KIND is a constant defined in mpif.h
+and gives the length of the declared integer in bytes.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+.SH SEE ALSO
+.ft R
+.sp
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_delete_attr.3

@@ -0,0 +1,58 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_delete_attr 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_delete_attr\fP \- Deletes attribute value associated with a key.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_delete_attr(MPI_Comm \fIcomm\fP, int \fIcomm_keyval\fP)
+
+.fi
+.SH INPUT/OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator from which the attribute is deleted (handle).
+
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm_keyval
+Key value (integer).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_delete_attr deletes an attribute from cache by key. This function invokes the attribute delete function delete_fn specified when the \fIcomm_keyval\fP was created. The call will fail if the delete_fn function returns an error code other than MPI_SUCCESS.
+
+Whenever a communicator is replicated using the function MPI_Comm_dup, all callback copy functions for attributes that are currently set are invoked (in arbitrary order). Whenever a communicator is deleted using the function MPI_Comm_free, all callback delete functions for attributes that are currently set are invoked.
+.sp
+This function is the same as MPI_Attr_delete but is needed to match the communicator-specific functions introduced in the MPI-2 standard. The use of MPI_Attr_delete is deprecated.
+
+
+.SH NOTES
+Note that it is not defined by the MPI standard what happens if the
+delete_fn callback invokes other MPI functions.  In Open MPI, it is
+not valid for delete_fn callbacks (or any of their children) to add or
+delete attributes on the same object on which the delete_fn callback
+is being invoked.
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_disconnect.3

@@ -0,0 +1,56 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_disconnect 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_disconnect\fP \- Deallocates communicator object and sets handle to MPI_COMM_NULL.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_disconnect(MPI_Comm *\fIcomm\fP)
+
+.fi
+.SH INPUT/OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_disconnect waits for all pending communication on \fIcomm\fP to complete internally, deallocates the communicator object, and sets the handle to MPI_COMM_NULL. It is a collective operation.
+.sp
+It may not be called with the communicator MPI_COMM_WORLD or MPI_COMM_SELF.
+.sp
+MPI_Comm_disconnect may be called only if all communication is complete and matched, so that buffered data can be delivered to its destination. This requirement is the same as for MPI_Finalize.
+.sp
+MPI_Comm_disconnect has the same action as MPI_Comm_free, except that it waits for pending communication to finish internally and enables the guarantee about the behavior of disconnected processes.
+
+.SH NOTES
+.ft R
+To disconnect two processes you may need to call MPI_Comm_disconnect, MPI_Win_free, and MPI_File_close to remove all communication paths between the two processes. Note that it may be necessary to disconnect several communicators (or to free several windows or files) before two processes are completely independent.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Comm_connect
+.br
+MPI_Comm_accept
+.br

macx64/mpi/openmpi/share/man/man3/MPI_Comm_dup.3

@@ -0,0 +1,62 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_dup 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_dup \fP \-  Duplicates an existing communicator with all its cached information.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_dup(MPI_Comm \fIcomm\fP, MPI_Comm\fI *newcomm\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+Copy of comm (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_dup duplicates the existing communicator comm with associated key
+values. For each key value, the respective copy callback function determines the attribute value associated with this key in the new communicator; one particular action that a copy callback may take is to delete the attribute from the new communicator. Returns in newcomm a new communicator with the same group, any copied cached information, but a new context (see Section 5.7.1 of the MPI-1 Standard, "Functionality").
+
+.SH NOTES
+This operation is used to provide a parallel
+library call with a duplicate communication space that has the same properties as the original communicator. This includes any attributes (see below) and topologies (see Chapter 6, "Process Topologies," in the MPI-1 Standard). This call is valid even if there are pending point-to-point communications involving the communicator comm. A typical call might involve an MPI_Comm_dup at the beginning of the parallel call, and an MPI_Comm_free of that duplicated communicator at the end of the call. Other models of communicator management are also possible.
+.sp
+This call applies to both intra- and intercommunicators.
+
+Note that it is not defined by the MPI standard what happens if the
+attribute copy callback invokes other MPI functions.  In Open MPI, it
+is not valid for attribute copy callbacks (or any of their children)
+to add or delete attributes on the same object on which the attribute
+copy callback is being invoked.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Comm_dup_with_info
+MPI_Comm_idup

macx64/mpi/openmpi/share/man/man3/MPI_Comm_dup_with_info.3

@@ -0,0 +1,71 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_dup_with_info 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_dup_with_info \fP \-  Duplicates an existing communicator using provided info.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_dup_with_info(MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP, MPI_Comm\fI *newcomm\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+.ft R
+.TP 1i
+info
+Info argument (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+Copy of comm (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_dup_with_info acts exactly like MPI_Comm_dup except that the
+info hints associated with the communicator \fIcomm\fP are not duplicated in \fInewcomm\fP. The
+hints provided by the argument \fIinfo\fP are associated with the output communicator \fInewcomm\fP
+instead.
+.sp
+See
+.BR MPI_Comm_set_info (3)
+for the list of recognized info keys.
+
+.SH NOTES
+This operation is used to provide a parallel
+library call with a duplicate communication space that has the same properties as the original communicator. This includes any attributes (see below) and topologies (see Chapter 6, "Process Topologies," in the MPI-1 Standard). This call is valid even if there are pending point-to-point communications involving the communicator comm. A typical call might involve an MPI_Comm_dup_with_info at the beginning of the parallel call, and an MPI_Comm_free of that duplicated communicator at the end of the call. Other models of communicator management are also possible.
+.sp
+This call applies to both intra- and intercommunicators.
+
+Note that it is not defined by the MPI standard what happens if the
+attribute copy callback invokes other MPI functions.  In Open MPI, it
+is not valid for attribute copy callbacks (or any of their children)
+to add or delete attributes on the same object on which the attribute
+copy callback is being invoked.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+MPI_Comm_dup
+MPI_Comm_idup
+MPI_Comm_set_info

macx64/mpi/openmpi/share/man/man3/MPI_Comm_f2c.3

@@ -0,0 +1,48 @@
+.\" -*- nroff -*-
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_f2c 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_f2c, MPI_Comm_c2f, MPI_File_f2c, MPI_File_c2f, MPI_Info_f2c, MPI_Info_c2f, MPI_Message_f2c, MPI_Message_c2f, MPI_Op_f2c, MPI_Op_c2f, MPI_Request_f2c, MPI_Request_c2f, MPI_Type_f2c, MPI_Type_c2f, MPI_Win_f2c, MPI_Win_c2f \fP \- Translates a C handle into a Fortran handle, or vice versa.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+MPI_Comm MPI_Comm_f2c(MPI_Fint \fIcomm\fP)
+MPI_Fint MPI_Comm_c2f(MPI_Comm \fIcomm\fP)
+
+MPI_File MPI_File_f2c(MPI_Fint \fIfile\fP)
+MPI_Fint MPI_File_c2f(MPI_File \fIfile\fP)
+
+MPI_Group MPI_Group_f2c(MPI Fint \fIgroup\fP)
+MPI_Fint MPI_Group_c2f(MPI Group \fIgroup\fP)
+
+MPI_Info MPI_Info_f2c(MPI_Fint \fIinfo\fP)
+MPI_Fint MPI_Info_c2f(MPI_Info \fIinfo\fP)
+
+MPI_Message MPI_Message_f2c(MPI_Fint \fImessage\fP)
+MPI_Fint MPI_Message_c2f(MPI_Message \fImessage\fP)
+
+MPI_Op MPI_Op_f2c(MPI_Fint \fIop\fP)
+MPI_Fint MPI_Op_c2f(MPI_Op \fIop\fP)
+
+MPI_Request MPI_Request_f2c(MPI_Fint \fIrequest\fP)
+MPI_Fint MPI_Request_c2f(MPI_Request \fIrequest\fP)
+
+MPI_Datatype MPI_Type_f2c(MPI_Fint \fIdatatype\fP)
+MPI_Fint MPI_Type_c2f(MPI_Datatype \fIdatatype\fP)
+
+MPI_Win MPI_Win_f2c(MPI_Fint \fIwin\fP)
+MPI_Fint MPI_Win_c2f(MPI_Win \fIwin\fP)
+
+.fi
+.SH DESCRIPTION
+.ft R
+Handles are passed between Fortran and C or C++ by using an explicit C wrapper to convert Fortran handles to C handles. There is no direct access to C or C++ handles in Fortran. Handles are passed between C and C++ using overloaded C++ operators called from C++ code. There is no direct access to C++ objects from C. The type definition \fIMPI_Fint\fP is provided in C/C++ for an integer of the size that matches a Fortran \fIINTEGER\fP; usually, \fIMPI_Fint\fP will be equivalent to \fIint\fP. The handle translation functions are provided in C to convert from a Fortran handle (which is an integer) to a C handle, and vice versa.
+.PP
+For example, if \fIcomm\fP is a valid Fortran handle to a communicator, then MPI_Comm_f2c returns a valid C handle to that same communicator; if \fIcomm\fP = MPI_COMM_NULL (Fortran value), then MPI_Comm_f2c returns a null C handle; if \fIcomm\fP is an invalid Fortran handle, then MPI_Comm_f2c returns an invalid C handle.
+.SH NOTE
+This function does not return an error value. Consequently, the result of calling it before MPI_Init or after MPI_Finalize is undefined.

macx64/mpi/openmpi/share/man/man3/MPI_Comm_free.3

@@ -0,0 +1,52 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_free 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_free \fP \- Mark a communicator object for deallocation.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_free(MPI_Comm *\fIcomm\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator to be destroyed (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This operation marks the communicator object for deallocation. The handle is set to MPI_COMM_NULL. Any pending operations that use this communicator will complete normally; the object is actually deallocated only if there are no other active references to it. This call applies to intracommunicators and intercommunicators. Upon actual deallocation, the delete callback functions for all cached attributes (see Section 5.7 in the MPI-1 Standard, "Caching") are called in arbitrary order.
+
+
+.SH NOTES
+Note that it is not defined by the MPI standard what happens if the
+delete_fn callback invokes other MPI functions.  In Open MPI, it is
+not valid for delete_fn callbacks (or any of their children) to add or
+delete attributes on the same object on which the delete_fn callback
+is being invoked.
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Comm_delete_attr

macx64/mpi/openmpi/share/man/man3/MPI_Comm_free_keyval.3

@@ -0,0 +1,46 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_free_keyval 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_free_keyval\fP \- Frees attribute key for communicator cache attribute.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_free_keyval(int *\fIcomm_keyval\fP)
+
+.fi
+.SH INPUT/OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm_keyval
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_free_keyval frees an extant attribute key. This function sets the value of \fIkeyval\fP to  MPI_KEYVAL_INVALID. Note that it is not erroneous to free an attribute key that is in use, because the actual free does not transpire until after all references (in other communicators on the process) to the key have been freed. These references need to be explicitly freed by the program, either via calls to MPI_Comm_delete_attr that free one attribute instance, or by calls to MPI_Comm_free that free all attribute instances associated with the freed communicator.
+.sp
+This call is identical to the call MPI_Keyval_free but is needed to match the  communicator-specific creation function introduced in the MPI-2 standard. The use of MPI_Keyval_free is deprecated.
+
+
+.SH NOTES
+.ft R
+Key values are global (they can be used with any and all communicators).
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_get_attr.3

@@ -0,0 +1,64 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_get_attr 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_get_attr\fP \- Retrieves attribute value by key.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_get_attr(MPI_Comm \fIcomm\fP, int \fIcomm_keyval\fP,
+	void *\fIattribute_val\fP, int *\fIflag\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator to which the attribute is attached (handle).
+.TP 1i
+comm_keyval
+Key value (integer).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+attribute_val
+Attribute value, unless f\fIlag\fP = false.
+.TP 1i
+flag
+False if no attribute is associated with the key (logical).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_get_attr retrieves an attribute value by key. The call is erroneous if there is no key with value \fIkeyval\fP. On the other hand, the call is correct if the key value exists, but no attribute is attached on \fIcomm\fP for that key; in that case, the call returns \fIflag\fP = false. In particular, MPI_KEYVAL_INVALID is an erroneous key value.
+.sp
+This function replaces MPI_Attr_get, the use of which is deprecated. The C binding is identical. The Fortran binding differs in that \fIattribute_val\fP is an address-sized integer.
+
+.SH FORTRAN 77 NOTES
+.ft R
+The MPI standard prescribes portable Fortran syntax for
+the \fIATTRIBUTE_VAL\fP argument only for Fortran 90. Sun FORTRAN 77
+users may use the non-portable syntax
+.sp
+.nf
+     INTEGER*MPI_ADDRESS_KIND \fIATTRIBUTE_VAL\fP
+.fi
+.sp
+where MPI_ADDRESS_KIND is a constant defined in mpif.h
+and gives the length of the declared integer in bytes.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_get_errhandler.3

@@ -0,0 +1,47 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_get_errhandler 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_get_errhandler \fP \- Retrieves error handler associated with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_get_errhandler(MPI_Comm \fIcomm\fP,
+	MPI_Errhandler *\fIerrhandler\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+errhandler
+New error handler for communicator (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_get_errhandler retrieves the error handler currently associated with a communicator. This call is identical to MPI_Errhandler_get, the use of which is deprecated.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_get_info.3

@@ -0,0 +1,59 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_get_info 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_get_info\fP \- Retrieves active communicator info hints
+.
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_get_info(MPI_Comm \fIcomm\fP, MPI_Info \fI*info_used\fP)
+.
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator from which to receive active info hints
+.
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+info_used
+New info object returned with all active hints on this communicator.
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+.
+.SH DESCRIPTION
+.ft R
+MPI_Comm_get_info returns a new info object containing the hints of
+the communicator associated with
+.IR comm .
+The current setting of all hints actually used by the system related
+to this communicator is returned in
+.IR info_used .
+If no such hints exist, a handle to a newly created info object is
+returned that contains no key/value pair. The user is responsible for
+freeing info_used via MPI_Info_free.
+.
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value
+of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler
+MPI_ERRORS_RETURN may be used to cause error values to be
+returned. Note that MPI does not guarantee that an MPI program can
+continue past an error.
+.
+.SH SEE ALSO
+MPI_Comm_get_info,
+MPI_Info_free

macx64/mpi/openmpi/share/man/man3/MPI_Comm_get_name.3

@@ -0,0 +1,54 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_get_name 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_get_name\fP \- Returns the name that was most recently associated with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_get_name(MPI_Comm \fIcomm\fP, char *\fIcomm_name\fP, int *\fIresultlen\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator the name of which is to be returned (handle).
+.TP 1i
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm_name
+Name previously stored on the communicator, or an empty string if no such name exists (string).
+.TP 1i
+resultlen
+Length of returned name (integer).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_get_name returns the last name that was previously associated with the given communicator. The name may be set and retrieved from any language. The same name will be returned independent of the language used. \fIcomm_name\fP should be allocated so that it can hold a resulting string of length MPI_MAX_OBJECT_NAME characters. MPI_Comm_get_name returns a copy of the set name in \fIcomm_name\fP.
+.sp
+If the user has not associated a name with a communicator, or an error occurs, MPI_Comm_get_name will return an empty string (all spaces in Fortran, "" in C and C++). The three predefined communicators will have predefined names associated with them. Thus, the names of MPI_COMM_WORLD, MPI_COMM_SELF, and MPI_COMM_PARENT will have the default of MPI_COMM_WORLD, MPI_COMM_SELF, and MPI_COMM_PARENT. The fact that the system may have chosen to give a default name to a communicator does not prevent the user from setting a name on the same communicator; doing this removes the old name and assigns the new one.
+
+.SH NOTES
+.ft R
+It is safe simply to print the string returned by MPI_Comm_get_name, as it is always a valid string even if there was no name.
+.sp
+Note that associating a name with a communicator has no effect on the semantics of an MPI program, and will (necessarily) increase the store requirement of the program, since the names must be saved. Therefore, there is no requirement that users use these functions to associate names with communicators. However debugging and profiling MPI applications may be made easier if names are associated with communicators, since the debugger or profiler should then be able to present information in a less cryptic manner.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_get_parent.3

@@ -0,0 +1,51 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_get_parent 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_get_parent\fP \- Returns the parent intercommunicator of current spawned process.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_get_parent(MPI_Comm *\fIparent\fP)
+
+.fi
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+parent
+The parent communicator (handle).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+If a process was started with MPI_Comm_spawn or MPI_Comm_spawn_multiple, MPI_Comm_get_parent returns the "parent" intercommunicator of the current process. This parent intercommunicator is created implicitly inside of MPI_Init and is the same intercommunicator returned by the spawn call made in the parents.
+.sp
+If the process was not spawned, MPI_Comm_get_parent returns MPI_COMM_NULL.
+.sp
+After the parent communicator is freed or disconnected, MPI_Comm_get_parent returns MPI_COMM_NULL.
+
+.SH NOTES
+.ft R
+MPI_Comm_get_parent returns a handle to a single intercommunicator. Calling MPI_Comm_get_parent a second time returns a handle to the same intercommunicator. Freeing the handle with MPI_Comm_disconnect or MPI_Comm_free will cause other references to the intercommunicator to become invalid (dangling). Note that calling MPI_Comm_free on the parent communicator is not useful.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Comm_spawn
+MPI_Comm_spawn_multiple
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_group.3

@@ -0,0 +1,43 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_group 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_group \fP \- Returns the group associated with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_group(MPI_Comm \fIcomm\fP, MPI_Group *\fIgroup\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator.
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+group
+Group in communicator (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+If the communicator is an intercommunicator (enables communication between two groups of processes), this function returns the local group.  To return the remote group, use the MPI_Comm_remote_group function.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_idup.3

@@ -0,0 +1,67 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_idup 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_idup \fP \-  Start the nonblocking duplication of an existing communicator with all its cached information.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_idup(MPI_Comm \fIcomm\fP, MPI_Comm\fI *newcomm\fP, MPI_Request\fI *request\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+Copy of comm (handle).
+.ft R
+.TP 1i
+request
+Communication request (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_idup starts the nonblocking duplication of an existing communicator comm with associated key
+values. For each key value, the respective copy callback function determines the attribute value associated with this key in the new communicator; one particular action that a copy callback may take is to delete the attribute from the new communicator. Returns in newcomm a new communicator with the same group, any copied cached information, but a new context (see Section 5.7.1 of the MPI-1 Standard, "Functionality"). The communicator returned in \fInewcomm\fP will not be available until the request is complete.
+.sp
+The completion of a communicator duplication request can be determined by calling any of MPI_Wait, MPI_Waitany, MPI_Test, or MPI_Testany with the request returned by this function.
+
+.SH NOTES
+This operation is used to provide a parallel
+library call with a duplicate communication space that has the same properties as the original communicator. This includes any attributes (see below) and topologies (see Chapter 6, "Process Topologies," in the MPI-1 Standard). This call is valid even if there are pending point-to-point communications involving the communicator comm. A typical call might involve an MPI_Comm_idup at the beginning of the parallel call, and an MPI_Comm_free of that duplicated communicator at the end of the call. Other models of communicator management are also possible.
+.sp
+This call applies to both intra- and intercommunicators.
+
+Note that it is not defined by the MPI standard what happens if the
+attribute copy callback invokes other MPI functions.  In Open MPI, it
+is not valid for attribute copy callbacks (or any of their children)
+to add or delete attributes on the same object on which the attribute
+copy callback is being invoked.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+
+.SH SEE ALSO
+MPI_Comm_dup
+MPI_Comm_dup_with_info

macx64/mpi/openmpi/share/man/man3/MPI_Comm_join.3

@@ -0,0 +1,89 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_join 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+\fBMPI_Comm_join\fP \- Establishes communication between MPI jobs
+
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_join(int \fIfd\fP, MPI_Comm *\fIintercomm\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+fd
+socket file descriptor (socket).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+intercomm
+Intercommunicator between processes (handle).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_join creates an intercommunicator from the union of two MPI
+processes that are connected by a socket. \fIfd\fP is a file
+descriptor representing a socket of type SOCK_STREAM (a two-way
+reliable byte-stream connection). Nonblocking I/O and asynchronous
+notification via SIGIO must not be enabled for the socket. The socket
+must be in a connected state, and must be quiescent when MPI_Comm_join
+is called.
+.sp
+MPI_Comm_join must be called by the process at each end of the
+socket. It does not return until both processes have called
+MPI_Comm_join.
+
+.SH NOTES
+.ft R
+There are no MPI library calls for opening and manipulating a socket.
+The socket \fIfd\fP can be opened using standard socket API calls.
+MPI uses the socket to bootstrap creation of the intercommunicator,
+and for nothing else. Upon return, the file descriptor will be open
+and quiescent.
+.sp
+In a multithreaded process, the application must ensure that other
+threads do not access the socket while one is in the midst of
+calling MPI_Comm_join.
+.sp
+The returned communicator will contain the two processes connected by
+the socket, and may be used to establish MPI communication with
+additional processes, through the usual MPI communicator-creation
+mechanisms.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+.SH SEE ALSO
+.ft R
+.nf
+socket(3SOCKET)
+MPI_Comm_create
+MPI_Comm_group
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_rank.3

@@ -0,0 +1,55 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_rank 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_rank\fP \- Determines the rank of the calling process in the communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_rank(MPI_Comm \fIcomm\fP, int\fI *rank\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+rank
+Rank of the calling process in group of comm (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function gives the rank of the process in the
+particular communicator's group. It is equivalent to accessing the
+communicator's group with MPI_Comm_group, computing the rank using MPI_Group_rank, and then freeing the temporary group via MPI_Group_free.
+.sp
+Many programs will be written with the master-slave model, where one process (such as the rank-zero process) will play a supervisory role, and the other processes will serve as compute nodes. In this framework, MPI_Comm_size and MPI_Comm_rank are useful for determining the roles of the various processes of a communicator.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Comm_group
+MPI_Comm_size
+MPI_Comm_compare
+