mac mpi

macx64/mpi/openmpi/share/man/man3/MPI.3

@@ -0,0 +1,194 @@
+.\" -*- nroff -*-
+.\" Copyright 2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+MPI \- General information Open MPI 4.0.1.
+
+.SH MPI
+.ft R
+Open MPI is an open source implementation of MPI (message-passing
+interface), the industry-standard specification for writing
+message-passing programs. Message passing is a programming model that
+gives the programmer explicit control over interprocess communication.
+.sp
+The MPI specification was developed by the MPI Forum, a group of
+software developers, computer vendors, academics, and computer-science
+researchers whose goal was to develop a standard for writing
+message-passing programs that would be efficient, flexible, and
+portable.
+.sp
+The outcome, known as the MPI Standard, was first published in 1993;
+its most recent version (MPI-2) was published in July 1997. Open MPI
+1.2 includes all MPI 1.2-compliant and MPI 2-compliant routines.
+.sp
+For more information about Open MPI, see the following URL:
+.nf
+
+   http://www.open-mpi.org
+.fi
+.sp
+The MPI standards are available at the following URL:
+.nf
+
+    http://www.mpi-forum.org
+.fi
+
+.SH MAN PAGE SYNTAX
+.ft R
+Man pages for Open MPI and Open MPI I/O routines are named according to C syntax, that is, they begin with the prefix "MPI_", all in uppercase, and the first letter following the "MPI_" prefix is also uppercase. The rest of the letters in the routine are all lowercase, for example, "MPI_Comm_get_attr".
+
+.SH ENVIRONMENT
+.ft R
+To fine-tune your Open MPI environment, you can either use arguments to the \fImpirun\fP, \fIorterun\fP, or \fImpiexec\fP commands, or you can use MCA parameters.
+.sp
+For more information on arguments, see the orterun.1 man page.
+.sp
+For a complete listing of MCA parameters and their descriptions, issue the command \fIompi_info -h\fP or \fIompi_info -param all all\fP. This information also appears in the FAQ on the Open MPI web site at:
+.nf
+
+   http://www.open-mpi.org/faq/?category=tuning#mca-params
+.fi
+
+.SH ERRORS
+.ft R
+All MPI routines (except MPI_Wtime and MPI_Wtick) return an error value; C routines as the value of the function and Fortran routines in the last
+argument.  Before the value is returned, the current MPI error handler is
+called.  By default, this error handler aborts the MPI job.  The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned.
+Note that MPI does not guarantee that an MPI program can continue past
+an error.
+.sp
+For more information on Open MPI error codes, see \fImpi.h\fP in the \fIinclude\fP directory.
+.sp
+Standard error return classes for Open MPI:
+.sp
+.nf
+MPI_SUCCESS                0      Successful return code.
+MPI_ERR_BUFFER             1      Invalid buffer pointer.
+MPI_ERR_COUNT              2      Invalid count argument.
+MPI_ERR_TYPE               3      Invalid datatype argument.
+MPI_ERR_TAG                4      Invalid tag argument.
+MPI_ERR_COMM               5      Invalid communicator.
+
+MPI_ERR_RANK               6      Invalid rank.
+MPI_ERR_REQUEST            7      Invalid MPI_Request handle.
+MPI_ERR_ROOT               8      Invalid root.
+MPI_ERR_GROUP              9      Null group passed to function.
+MPI_ERR_OP                10      Invalid operation.
+
+MPI_ERR_TOPOLOGY          11      Invalid topology.
+MPI_ERR_DIMS              12      Illegal dimension argument.
+MPI_ERR_ARG               13      Invalid argument.
+MPI_ERR_UNKNOWN           14      Unknown error.
+MPI_ERR_TRUNCATE          15      Message truncated on receive.
+
+MPI_ERR_OTHER             16      Other error; use Error_string.
+MPI_ERR_INTERN            17      Internal error code.
+MPI_ERR_IN_STATUS         18      Look in status for error value.
+MPI_ERR_PENDING           19      Pending request.
+MPI_ERR_ACCESS            20      Permission denied.
+
+MPI_ERR_AMODE             21      Unsupported amode passed to open.
+MPI_ERR_ASSERT            22      Invalid assert.
+MPI_ERR_BAD_FILE          23      Invalid file name (for example,
+                                  path name too long).
+MPI_ERR_BASE              24      Invalid base.
+MPI_ERR_CONVERSION        25      An error occurred in a user-supplied
+                                  data-conversion function.
+
+MPI_ERR_DISP              26      Invalid displacement.
+MPI_ERR_DUP_DATAREP       27      Conversion functions could not
+                                  be registered because a data
+                                  representation identifier that was
+                                  already defined was passed to
+                                  MPI_REGISTER_DATAREP.
+MPI_ERR_FILE_EXISTS       28      File exists.
+MPI_ERR_FILE_IN_USE       29      File operation could not be
+                                  completed, as the file is currently
+                                  open by some process.
+MPI_ERR_FILE              30      Invalid file handle.
+
+MPI_ERR_INFO_KEY          31      Illegal info key.
+MPI_ERR_INFO_NOKEY        32      No such key.
+MPI_ERR_INFO_VALUE        33      Illegal info value.
+MPI_ERR_INFO              34      Invalid info object.
+MPI_ERR_IO                35      I/O error.
+
+MPI_ERR_KEYVAL            36      Illegal key value.
+MPI_ERR_LOCKTYPE          37      Invalid locktype.
+MPI_ERR_NAME              38      Name not found.
+MPI_ERR_NO_MEM            39      Memory exhausted.
+MPI_ERR_NOT_SAME          40      Collective argument not identical
+                                  on all processes, or collective
+                                  routines called in a different order
+                                  by different processes.
+
+MPI_ERR_NO_SPACE          41      Not enough space.
+MPI_ERR_NO_SUCH_FILE      42      File (or directory) does not exist.
+MPI_ERR_PORT              43      Invalid port.
+MPI_ERR_QUOTA             44      Quota exceeded.
+MPI_ERR_READ_ONLY         45      Read-only file system.
+
+MPI_ERR_RMA_CONFLICT      46      Conflicting accesses to window.
+MPI_ERR_RMA_SYNC          47      Erroneous RMA synchronization.
+MPI_ERR_SERVICE           48      Invalid publish/unpublish.
+MPI_ERR_SIZE              49      Invalid size.
+MPI_ERR_SPAWN             50      Error spawning.
+
+MPI_ERR_UNSUPPORTED_DATAREP
+                          51      Unsupported datarep passed to
+                                  MPI_File_set_view.
+MPI_ERR_UNSUPPORTED_OPERATION
+                          52      Unsupported operation, such as
+                                  seeking on a file that supports
+                                  only sequential access.
+MPI_ERR_WIN               53      Invalid window.
+MPI_T_ERR_MEMORY          54      Out of memory.
+MPI_T_ERR_NOT_INITIALIZED 55      Interface not initialized.
+
+MPI_T_ERR_CANNOT_INIT     56      Interface not in the state to be
+                                  initialized.
+MPI_T_ERR_INVALID_INDEX   57      The enumeration index is invalid.
+MPI_T_ERR_INVALID_ITEM    58      The item index queried is out of
+                                  range.
+MPI_T_ERR_INVALID_HANDLE  59      The handle is invalid.
+MPI_T_ERR_OUT_OF_HANDLES  60      No more handles available.
+
+MPI_T_ERR_OUT_OF_SESSIONS 61      No more sessions available.
+MPI_T_ERR_INVALID_SESSION 62      Session argument is not a valid
+                                  session.
+MPI_T_ERR_CVAR_SET_NOT_NOW
+                          63      Variable cannot be set at this
+                                  moment.
+MPI_T_ERR_CVAR_SET_NEVER  64      Variable cannot be set until end of
+                                  execution.
+MPI_T_ERR_PVAR_NO_STARTSTOP
+                          65      Variable cannot be started or stopped.
+
+MPI_T_ERR_PVAR_NO_WRITE   66      Variable cannot be written or reset.
+MPI_T_ERR_PVAR_NO_ATOMIC  67      Variable cannot be read and written
+                                  atomically.
+MPI_ERR_RMA_RANGE         68      Target memory is not part of the
+                                  window (in the case of a window
+                                  created with MPI_WIN_CREATE_DYNAMIC,
+                                  target memory is not attached).
+MPI_ERR_RMA_ATTACH        69      Memory cannot be attached (e.g.,
+                                  because of resource exhaustion).
+MPI_ERR_RMA_FLAVOR        70      Passed window has the wrong flavor
+                                  for the called function.
+
+MPI_ERR_RMA_SHARED        71      Memory cannot be shared (e.g., some
+                                  process in the group of the specified
+                                  communicator cannot expose shared
+                                  memory).
+MPI_T_ERR_INVALID         72      Invalid use of the interface or bad
+                                  parameter values(s).
+MPI_T_ERR_INVALID_NAME    73      The variable or category name is
+                                  invalid.
+
+MPI_ERR_LASTCODE          93      Last error code.
+.fi
+

macx64/mpi/openmpi/share/man/man3/MPIX_Allgather_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Allgatherv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Allreduce_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Alltoall_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Alltoallv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Alltoallw_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Barrier_init.3

@@ -0,0 +1,156 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2018      FUJITSU LIMITED.  All rights reserved.
+.\" $COPYRIGHT$
+.TH MPIX_Barrier_init 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPIX_Allgather_init, MPIX_Allgatherv_init, MPIX_Allreduce_init, MPIX_Alltoall_init, MPIX_Alltoallv_init, MPIX_Alltoallw_init, MPIX_Barrier_init, MPIX_Bcast_init, MPIX_Exscan_init, MPIX_Gather_init, MPIX_Gatherv_init, MPIX_Reduce_init, MPIX_Reduce_scatter_init, MPIX_Reduce_scatter_block_init, MPIX_Scan_init, MPIX_Scatter_init, MPIX_Scatterv_init, MPIX_Neighbor_allgather_init, MPIX_Neighbor_allgatherv_init, MPIX_Neighbor_alltoall_init, MPIX_Neighbor_alltoallv_init, MPIX_Neighbor_alltoallw_init\fP \- Builds a handle for a collective communication or neighborhood collective communication
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+#include <mpi-ext.h>
+int MPIX_Allgather_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Allgatherv_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[],
+	const int \fIdispls\fP[], MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Allreduce_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP, int \fIcount\fP,
+	MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Alltoall_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Alltoallv_init(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP,
+	const int \fIrecvcounts\fP[], const int \fIrdispls\fP[], MPI_Datatype \fIrecvtype\fP,
+	MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Alltoallw_init(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], const MPI_Datatype \fIsendtypes\fP[], void *\fIrecvbuf\fP,
+	const int \fIrecvcounts\fP[], const int \fIrdispls\fP[],
+	const MPI_Datatype \fIrecvtypes\fP[], MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Barrier_init(MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Bcast_init(void *\fIbuffer\fP, int \fIcount\fP, MPI_Datatype \fIdatatype\fP,
+	int \fIroot\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Exscan_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP, int \fIcount\fP,
+	MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Gather_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Gatherv_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[],
+	const int \fIdispls\fP[], MPI_Datatype \fIrecvtype\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Reduce_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP, int \fIcount\fP,
+	MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Reduce_scatter_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP,
+	const int \fIrecvcounts\fP[], MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP,
+	MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Reduce_scatter_block_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP,
+	int \fIrecvcount\fP, MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Scan_init(const void *\fIsendbuf\fP, void *\fIrecvbuf\fP, int \fIcount\fP,
+	MPI_Datatype \fIdatatype\fP, MPI_Op \fIop\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Scatter_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Scatterv_init(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIdispls\fP[], MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP,
+	int \fIrecvcount\fP, MPI_Datatype \fIrecvtype\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Neighbor_allgather_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Neighbor_allgatherv_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[],
+	const int \fIdispls\fP[], MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP,
+	MPI_Info \fIinfo\fP, MPI_Request *\fIrequest\fP)
+
+int MPIX_Neighbor_alltoall_init(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Neighbor_alltoallv_init(const void *\fIsendbuf\fP,
+	const int \fIsendcounts\fP[], const int \fIsdispls\fP[], MPI_Datatype \fIsendtype\fP,
+	void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[], const int \fIrdispls\fP[],
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+int MPIX_Neighbor_alltoallw_init(const void *\fIsendbuf\fP,
+	const int \fIsendcounts\fP[], const MPI_Aint \fIsdispls\fP[],
+	const MPI_Datatype \fIsendtypes\fP[], void *\fIrecvbuf\fP,
+	const int \fIrecvcounts\fP[], const MPI_Aint \fIrdispls\fP[],
+	const MPI_Datatype \fIrecvtypes\fP[], MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP,
+	MPI_Request *\fIrequest\fP)
+
+.fi
+.SH DESCRIPTION
+.ft R
+Creates a persistent communication request for a collective operation or neighborhood collective operation.
+
+As of Nov. 2018, the feature of persistent collective communication operations and persistent collective neighborhood communication operations is planned to be included in the next MPI Standard after MPI-3.1.
+.nf
+
+    https://github.com/mpi-forum/mpi-issues/issues/25
+.fi
+
+Open MPI implements 2018 Draft Specification of the MPI standard shown in the following URL.
+.nf
+
+    https://www.mpi-forum.org/docs/drafts/mpi-2018-draft-report.pdf
+.fi
+
+The interface may still change in the standard. Therefore the prefix \fIMPIX_\fP is used instead of \fIMPI_\fP for these request creation routines. To start, complete, and free the created request, usual MPI routines (\fIMPI_Start\fP etc.) can be used.
+
+Future versions of Open MPI will switch to the \fIMPI_\fP prefix and will not require the C header file \fImpi-ext.h\fP, the Fortran modules \fImpi_ext\fP and \fImpi_f08_ext\fP, and the Fortran header file \fImpif-ext.h\fP once the MPI Standard which includes this feature is published.
+
+.SH EXAMPLE
+.nf
+
+    MPI_Request req;
+    MPIX_Barrier_init(MPI_COMM_WORLD, MPI_INFO_NULL, &req);
+    MPI_Start(&req);
+    MPI_Wait(&req, MPI_STATUS_IGNORE);
+    MPI_Request_free(&req);
+.fi
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Start
+.br
+MPI_Startall
+.br
+MPI_Request_free

macx64/mpi/openmpi/share/man/man3/MPIX_Bcast_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Exscan_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Gather_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Gatherv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Neighbor_allgather_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Neighbor_allgatherv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Neighbor_alltoall_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Neighbor_alltoallv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Neighbor_alltoallw_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Query_cuda_support.3

@@ -0,0 +1,28 @@
+.\" Copyright 2007-2010 Oracle and/or its affiliates.  All rights reserved.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" Copyright (c) 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright (c) 2015 NVIDIA, Inc. All rights reserved.
+.TH MPIx_CUDA_SUPPORT 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPIX_Query_cuda_support\fP \- Returns 1 if there is CUDA aware support and 0 if there is not.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+#include <mpi-ext.h>
+
+int MPIX_Query_cuda_support(void)
+.fi
+.SH DESCRIPTION
+.ft R
+
+.SH Examples
+.ft R
+
+.SH See Also
+.ft R
+.nf
+
+.fi

macx64/mpi/openmpi/share/man/man3/MPIX_Reduce_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Reduce_scatter_block_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Reduce_scatter_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Scan_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Scatter_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPIX_Scatterv_init.3

@@ -0,0 +1 @@
+.so man3/MPIX_Barrier_init.3

macx64/mpi/openmpi/share/man/man3/MPI_Abort.3

@@ -0,0 +1,51 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Abort 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Abort\fP \- Terminates MPI execution environment.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Abort(MPI_Comm \fIcomm\fP, int\fI errorcode\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator of tasks to abort.
+.TP 1i
+errorcode
+Error code to return to invoking environment.
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This routine makes a "best attempt" to abort all tasks in the group of
+comm. This function does not require that the invoking environment take any
+action with the error code. However, a UNIX or POSIX
+environment should handle this as a return errorcode from the main program
+or an abort (errorcode).
+.sp
+The long-term goal of the Open MPI implementation is to terminate all processes in all tasks that contain a process in \fIcomm\fP, and the error code is not returned to the invoking environment. At the moment, this isn't fully implemented and MPI_Abort will terminate the entire job.
+.sp
+Note: All associated processes are sent a SIGTERM.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Accumulate.3

@@ -0,0 +1,119 @@
+.\" -*- nroff -*-
+.\" Copyright 2013-2014 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Accumulate 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Accumulate\fP, \fBMPI_Raccumulate\fP \- Combines the contents of the origin buffer with that of a target buffer.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Accumulate(const void *\fIorigin_addr\fP, int \fIorigin_count\fP,
+	MPI_Datatype \fIorigin_datatype\fP, int \fItarget_rank\fP,
+	MPI_Aint \fItarget_disp\fP, int \fItarget_count\fP,
+	MPI_Datatype \fItarget_datatype\fP, MPI_Op \fIop\fP, MPI_Win \fIwin\fP)
+
+int MPI_Raccumulate(const void *\fIorigin_addr\fP, int \fIorigin_count\fP,
+	MPI_Datatype \fIorigin_datatype\fP, int \fItarget_rank\fP,
+	MPI_Aint \fItarget_disp\fP, int \fItarget_count\fP,
+	MPI_Datatype \fItarget_datatype\fP, MPI_Op \fIop\fP, MPI_Win \fIwin\fP,
+	MPI_Request *\fIrequest\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+origin_addr
+Initial address of buffer (choice).
+.ft R
+.TP 1i
+origin_count
+Number of entries in buffer (nonnegative integer).
+.ft R
+.TP 1i
+origin_datatype
+Data type of each buffer entry (handle).
+.ft R
+.TP 1i
+target_rank
+Rank of target (nonnegative integer).
+.ft R
+.TP 1i
+target_disp
+Displacement from start of window to beginning of target buffer (nonnegative integer).
+.ft R
+.TP 1i
+target_count
+Number of entries in target buffer (nonnegative integer).
+.ft R
+.TP 1i
+target_datatype
+Data type of each entry in target buffer (handle).
+.ft R
+.TP 1i
+op
+Reduce operation (handle).
+.ft R
+.TP 1i
+win
+Window object (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+MPI_Raccumulate: RMA request
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+\fBMPI_Accumulate\fP is a function used for one-sided MPI communication that adds the contents of the origin buffer (as defined by \fIorigin_addr\fP, \fIorigin_count\fP, and \fIorigin_datatype\fP) to the buffer specified by the arguments \fItarget_count\fP and \fItarget_datatype\fP, at offset \fItarget_disp\fP, in the target window specified by \fItarget_rank\fP and \fIwin\fP, using the operation \fIop\fP. The target window can only be accessed by processes within the same node. This is similar to MPI_Put, except that data is combined into the target area instead of overwriting it.
+.sp
+Any of the predefined operations for MPI_Reduce can be used. User-defined functions cannot be used. For example, if \fIop\fP is MPI_SUM, each element of the origin buffer is added to the corresponding element in the target, replacing the former value in the target.
+.sp
+Each datatype argument must be a predefined data type or a derived data type, where all basic components are of the same predefined data type. Both datatype arguments must be constructed from the same predefined data type. The operation \fIop\fP applies to elements of that predefined type. The \fItarget_datatype\fP argument must not specify overlapping entries, and the target buffer must fit in the target window.
+.sp
+A new predefined operation, MPI_REPLACE, is defined. It corresponds to the associative function f(a, b) =b; that is, the current value in the target memory is replaced by the value supplied by the origin.
+.sp
+\fBMPI_Raccumulate\fP is similar to \fBMPI_Accumulate\fP, except that it allocates a communication request object and associates it with the request handle (the argument \fIrequest\fP) that can be used to wait or test for completion. The completion of an \fBMPI_Raccumulate\fP operation indicates that the \fIorigin_addr\fP buffer is free to be updated. It does not indicate that the operation has completed at the target window.
+
+.SH FORTRAN 77 NOTES
+.ft R
+The MPI standard prescribes portable Fortran syntax for
+the \fITARGET_DISP\fP argument only for Fortran 90.  FORTRAN 77
+users may use the non-portable syntax
+.sp
+.nf
+     INTEGER*MPI_ADDRESS_KIND \fITARGET_DISP\fP
+.fi
+.sp
+where MPI_ADDRESS_KIND is a constant defined in mpif.h
+and gives the length of the declared integer in bytes.
+
+.SH NOTES
+MPI_Put is a special case of \fBMPI_Accumulate\fP, with the operation MPI_REPLACE. Note, however, that MPI_Put and \fBMPI_Accumulate\fP have different constraints on concurrent updates.
+.sp
+It is the user's responsibility to guarantee that, when
+using the accumulate functions, the target displacement argument is such
+that accesses to the window are properly aligned according to the data
+type arguments in the call to the \fBMPI_Accumulate\fP function.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Put
+MPI_Get_accumulate
+MPI_Reduce

macx64/mpi/openmpi/share/man/man3/MPI_Add_error_class.3

@@ -0,0 +1,77 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Add_error_class 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+.nf
+\fBMPI_Add_error_class\fP \- Creates a new error class and returns its value
+
+.fi
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Add_error_class(int *\fIerrorclass\fP)
+
+.fi
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.4i
+errorclass
+New error class (integer).
+.ft R
+.TP 1.4i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+The function MPI_Add_error_class creates a new, local error
+class.
+
+.SH NOTES
+.ft R
+Because this function is local, the same value of \fIerrorclass\fP may
+not be returned on all processes that make this call, even if they
+call the function concurrently. Thus, same error on different
+processes may not cause the same value of \fIerrorclass\fP to be
+returned. To reduce the potential for confusion, MPI_Add_error_string
+may be used on multiple processes to associate the same error string
+with the newly created \fIerrorclass\fP. Even though \fIerrorclass\fP
+may not be consistent across processes, using MPI_Add_error_string
+will ensure the error string associated with it will be the same
+everywhere.
+.sp
+No function is provided to free error classes, as it is not expected
+that an application will create them in significant numbers.
+.sp
+The value returned is always greater than or equal to MPI_ERR_LASTCODE.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Add_error_code
+MPI_Add_error_string
+MPI_Error_class
+MPI_Error_string
+

macx64/mpi/openmpi/share/man/man3/MPI_Add_error_code.3

@@ -0,0 +1,69 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Add_error_code 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+\fBMPI_Add_error_code\fP \- Creates a new error code associated
+with \fIerrorclass\fP
+
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Add_error_code(int \fIerrorclass\fP, int *\fIerrorcode\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1.4i
+errorclass
+MPI error class (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.4i
+errorcode
+Error code returned by an MPI routine or an MPI error class (integer).
+.ft R
+.TP 1.4i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+Creates a new error code associated with \fIerrorclass\fP and returns
+its value in \fIerrorcode\fP.
+
+.SH NOTES
+.ft R
+No function is provided to free error codes, as it is not expected
+that an application will create them in significant numbers.
+.sp
+The value returned is always greater than or equal to MPI_ERR_LASTCODE.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Add_error_class
+MPI_Error_class
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Add_error_string.3

@@ -0,0 +1,72 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Add_error_string 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+.nf
+\fBMPI_Add_error_string\fP \- Associates a string with an error code or class
+
+.fi
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Add_error_string(int \fIerrorcode\fP, const char *\fIstring\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1.4i
+errorcode
+MPI error class, or an error code returned by an MPI routine (integer).
+.ft R
+.TP 1.4i
+string
+Text that corresponds to the error code or class (string).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1.4i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This routine associates an error string with an error code or
+class. Calling MPI_Add_error_string for an error code or class that
+already has an associated error string will replace the old string
+with the new one. It is erroneous to call MPI_Add_error_string for an
+error value not generated via MPI_Add_error_class or
+MPI_Add_error_code (e.g., an error code or class with a value not
+greater than MPI_LAST_ERRCODE).
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Add_error_class
+MPI_Add_error_code
+MPI_Error_class
+MPI_Error_string
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Address.3

@@ -0,0 +1,84 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Address 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Address\fP \- Gets the address of a location in memory -- use of this routine is deprecated.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Address(void *\fIlocation\fP, MPI_Aint\fP *address\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+location
+Location in caller memory (choice).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+address
+Address of location (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Note that use of this routine is \fIdeprecated\fP as of MPI-2. Please use MPI_Get_address instead.
+.sp
+This deprecated routine is not available in C++.
+.sp
+The address of a location in memory can be found by invoking this function. Returns the (byte) address of location.
+.sp
+Example: Using MPI_Address for an array.
+.sp
+.nf
+REAL A(100,100)
+.fi
+.br
+   INTEGER I1, I2, DIFF
+.br
+   CALL MPI_ADDRESS(A(1,1), I1, IERROR)
+.br
+   CALL MPI_ADDRESS(A(10,10), I2, IERROR)
+.br
+   DIFF = I2 - I1
+.br
+! The value of DIFF is 909*sizeofreal; the values of I1 and I2 are
+.br
+! implementation dependent.
+.fi
+
+.SH NOTES
+.ft R
+This routine is provided for both Fortran and C programmers and may be useful when writing portable code. In the current release, the address returned by this routine will be the same as that produced by the C & operator.
+.sp
+C users may be tempted to avoid using
+MPI_Address and rely on the availability of the address operator &.
+Note, however, that & cast-expression is a pointer, not an address.
+ANSI C does not require that the value of a pointer (or the pointer
+cast to int) be the absolute address of the object pointed at although
+this is commonly the case. Furthermore, referencing may not have a unique
+definition on machines with a segmented address space. The use of
+MPI_Address to "reference" C variables guarantees portability to
+such machines as well.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+MPI_Get_address
+.br

macx64/mpi/openmpi/share/man/man3/MPI_Aint_add.3

@@ -0,0 +1,73 @@
+.\" -*- nroff -*-
+.\" Copyright 2013-2015 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Aint_add 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Aint_add\fP, \fBMPI_Aint_diff\fP \- Portable functions for
+arithmetic on MPI_Aint values.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+MPI_Aint MPI_Aint_add(MPI_Aint \fIbase\fP, MPI_Aint \fIdisp\fP)
+
+MPI_Aint MPI_Aint_diff(MPI_Aint \fIaddr1\fP, MPI_Aint \fIaddr2\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+base
+Base address (integer).
+.ft R
+.TP 1i
+disp
+Displacement (integer).
+.ft R
+.TP 1i
+addr1
+Minuend address (integer).
+.ft R
+.TP
+addr2
+Subtrahend address (integer).
+
+.SH DESCRIPTION
+.ft R
+\fBMPI_Aint_add\fP produces a new MPI_Aint value that is equivalent to the sum of
+the \fIbase\fP and \fIdisp\fP arguments, where \fIbase\fP represents
+a base address returned by a call to \fBMPI_Get_address\fP and
+\fIdisp\fP represents a signed integer displacement. The resulting
+address is valid only at the process that generated \fIbase\fP, and it
+must correspond to a location in the same object referenced by
+\fIbase\fP, as described in MPI-3.1 \[char167] 4.1.12. The addition is
+performed in a manner that results in the correct MPI_Aint
+representation of the output address, as if the process that
+originally produced \fIbase\fP had called:
+
+.nf
+        \fBMPI_Get_address\fP ((char *) \fIbase\fP + \fIdisp\fP, &\fIresult\fP);
+.fi
+.sp
+.ft R
+\fBMPI_Aint_diff\fP produces a new MPI_Aint value that is equivalent
+to the difference between \fIaddr1\fP and \fIaddr2\fP arguments, where
+\fIaddr1\fP and \fIaddr2\fP represent addresses returned by calls to
+\fBMPI_Get_address\fP. The resulting address is valid only at the
+process that generated \fIaddr1\fP and \fIaddr2\fP, and \fIaddr1\fP
+and \fIaddr2\fP must correspond to locations in the same object in the
+same process, as described in MPI-3.1 \[char167] 4.1.12. The difference is
+calculated in a manner that results in the signed difference from
+\fIaddr1\fP to \fIaddr2\fP, as if the process that originally produced
+the addresses had called (char *) \fIaddr1\fP - (char *) \fIaddr2\fP
+on the addresses initially passed to \fBMPI_Get_address\fP.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Get_address

macx64/mpi/openmpi/share/man/man3/MPI_Aint_diff.3

@@ -0,0 +1 @@
+.so man3/MPI_Aint_add.3

macx64/mpi/openmpi/share/man/man3/MPI_Allgather.3

@@ -0,0 +1,129 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Allgather 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Allgather, MPI_Iallgather\fP \- Gathers data from all processes and distributes it to all processes
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Allgather(const void\fI *sendbuf\fP, int \fI sendcount\fP,
+	 MPI_Datatype\fI sendtype\fP, void\fI *recvbuf\fP, int\fI recvcount\fP,
+	 MPI_Datatype\fI recvtype\fP, MPI_Comm\fI comm\fP)
+
+int MPI_Iallgather(const void\fI *sendbuf\fP, int \fI sendcount\fP,
+	 MPI_Datatype\fI sendtype\fP, void\fI *recvbuf\fP, int\fI recvcount\fP,
+	 MPI_Datatype\fI recvtype\fP, MPI_Comm\fI comm\fP, MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+sendbuf
+Starting address of send buffer (choice).
+.TP 1i
+sendcount
+Number of elements in send buffer (integer).
+.TP 1i
+sendtype
+Datatype of send buffer elements (handle).
+.TP 1i
+recvbuf
+Starting address of recv buffer (choice).
+.TP 1i
+recvcount
+Number of elements received from any process (integer).
+.TP 1i
+recvtype
+Datatype of receive buffer elements (handle).
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+recvbuf
+Address of receive buffer (choice).
+.ft R
+.TP 1i
+request
+Request (handle, non-blocking only).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Allgather is similar to MPI_Gather, except that all processes receive the result, instead of just the root. In other words, all processes contribute to the result, and all processes receive the result.
+.sp
+The type signature associated with sendcount, sendtype at a process must be equal to the type signature associated with recvcount, recvtype at any other process.
+.sp
+The outcome of a call to MPI_Allgather(\&...) is as if all processes executed n calls to
+.sp
+.nf
+  MPI_Gather(sendbuf,sendcount,sendtype,recvbuf,recvcount,
+             recvtype,root,comm),
+.fi
+.sp
+.fi
+for root = 0 , ..., n-1. The rules for correct usage of MPI_Allgather are easily found from the corresponding rules for MPI_Gather.
+.sp
+\fBExample:\fR The all-gather version of Example 1 in MPI_Gather. Using  MPI_Allgather, we will gather 100 ints from every process in the group to every process.
+.sp
+.nf
+MPI_Comm comm;
+    int gsize,sendarray[100];
+    int *rbuf;
+    \&...
+    MPI_Comm_size( comm, &gsize);
+    rbuf = (int *)malloc(gsize*100*sizeof(int));
+    MPI_Allgather( sendarray, 100, MPI_INT, rbuf, 100, MPI_INT, comm);
+.fi
+.sp
+After the call, every process has the group-wide concatenation of the sets of data.
+
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-gather operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR.  In this case, \fIsendcount\fR and \fIsendtype\fR are ignored.  The input data of each process is assumed to be in the area where that process would receive its own contribution to the receive buffer.  Specifically, the outcome of a call to MPI_Allgather that used the in-place option is identical to the case in which all processes executed \fIn\fR calls to
+.sp
+.nf
+   MPI_GATHER ( MPI_IN_PLACE, 0, MPI_DATATYPE_NULL, recvbuf,
+   recvcount, recvtype, root, comm )
+
+for root =0, ... , n-1.
+.fi
+.sp
+Note that MPI_IN_PLACE is a special kind of value; it has the same restrictions on its use as MPI_BOTTOM.
+.sp
+Because the in-place option converts the receive buffer into a send-and-receive buffer, a Fortran binding that includes INTENT must mark these as INOUT, not OUT.
+.sp
+.SH WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the gather operation occurs in two phases.  The data is gathered from all the members of the first group and received by all the members of the second group.  Then the data is gathered from all the members of the second group and received by all the members of the first.  The operation, however, need not be symmetric.  The number of items sent by the processes in first group need not be equal to the number of items sent by the the processes in the second group.  You can move data in only one direction by giving \fIsendcount\fR a value of 0 for communication in the reverse direction.
+.sp
+The first group defines the root process.  The root process uses MPI_ROOT as the value of \fIroot\fR.  All other processes in the first group use MPI_PROC_NULL as the value of \fIroot\fR.  All processes in the second group use the rank of the root process in the first group as the value of \fIroot\fR.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+.sp
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Allgatherv
+.br
+MPI_Gather
+

macx64/mpi/openmpi/share/man/man3/MPI_Allgatherv.3

@@ -0,0 +1,113 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2007-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Allgatherv 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Allgatherv, MPI_Iallgatherv\fP \- Gathers data from all processes and delivers it to all. Each process may contribute a different amount of data.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Allgatherv(const void\fI *sendbuf\fP, int\fI sendcount\fP,
+	MPI_Datatype\fI sendtype\fP, void\fI *recvbuf\fP, const int\fI recvcounts[]\fP,
+	const int\fI displs[]\fP, MPI_Datatype\fI recvtype\fP, MPI_Comm\fI comm\fP)
+
+int MPI_Iallgatherv(const void\fI *sendbuf\fP, int\fI sendcount\fP,
+	MPI_Datatype\fI sendtype\fP, void\fI *recvbuf\fP, const int\fI recvcounts[]\fP,
+	const int\fI displs[]\fP, MPI_Datatype\fI recvtype\fP, MPI_Comm\fI comm\fP,
+        MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+sendbuf
+Starting address of send buffer (choice).
+.TP 1i
+sendcount
+Number of elements in send buffer (integer).
+.TP 1i
+sendtype
+Datatype of send buffer elements (handle).
+.TP 1i
+recvcount
+Integer array (of length group size) containing the number of elements that are received from each process.
+.TP 1i
+displs
+Integer array (of length group size). Entry i specifies the displacement (relative to recvbuf) at which to place the incoming data from process i.
+.TP 1i
+recvtype
+Datatype of receive buffer elements (handle).
+.TP 1i
+comm
+Communicator (handle).
+.sp
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+recvbuf
+Address of receive buffer (choice).
+.TP 1i
+request
+Request (handle, non-blocking only).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Allgatherv is similar to MPI_Allgather in that all processes gather data from all other processes, except that each process can send a different amount of data. The block of data sent from the jth process is received by every process and placed in the jth block of the buffer
+.I recvbuf.
+.sp
+The type signature associated with sendcount, sendtype, at process j must be equal to the type signature associated with recvcounts[j], recvtype at any other process.
+.sp
+The outcome is as if all processes executed calls to
+.nf
+MPI_Gatherv(sendbuf,sendcount,sendtype,recvbuf,recvcount,
+            displs,recvtype,root,comm)
+.fi
+.sp
+for root = 0 , ..., n-1. The rules for correct usage of MPI_Allgatherv are easily found from the corresponding rules for MPI_Gatherv.
+
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-gather operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR.  In this case, \fIsendcount\fR and \fIsendtype\fR are ignored.  The input data of each process is assumed to be in the area where that process would receive its own contribution to the receive buffer.  Specifically, the outcome of a call to MPI_Allgather that used the in-place option is identical to the case in which all processes executed \fIn\fR calls to
+.sp
+.nf
+   MPI_GATHERV ( MPI_IN_PLACE, 0, MPI_DATATYPE_NULL, recvbuf,
+   recvcounts, displs, recvtype, root, comm )
+
+for root =0, ... , n-1.
+.fi
+.sp
+Note that MPI_IN_PLACE is a special kind of value; it has the same restrictions on its use as MPI_BOTTOM.
+.sp
+Because the in-place option converts the receive buffer into a send-and-receive buffer, a Fortran binding that includes INTENT must mark these as INOUT, not OUT.
+.sp
+.SH WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the gather operation occurs in two phases.  The data is gathered from all the members of the first group, concatenated, and received by all the members of the second group.  Then the data is gathered from all the members of the second group, concatenated, and received by all the members of the first.  The send buffer arguments in the one group must be consistent with the receive buffer arguments in the other group, and vice versa.  The operation must exhibit symmetric, full-duplex behavior.
+.sp
+The first group defines the root process.  The root process uses MPI_ROOT as the value of \fIroot\fR.  All other processes in the first group use MPI_PROC_NULL as the value of \fIroot\fR.  All processes in the second group use the rank of the root process in the first group as the value of \fIroot\fR.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+.sp
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Gatherv
+.br
+MPI_Allgather

macx64/mpi/openmpi/share/man/man3/MPI_Alloc_mem.3

@@ -0,0 +1,87 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Alloc_mem 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Alloc_mem \fP \- Allocates a specified memory segment.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Alloc_mem(MPI_Aint \fIsize\fP, MPI_Info \fIinfo\fP, void *\fIbaseptr\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+size
+Size of memory segment in bytes (nonnegative integer).
+.ft R
+.TP 1i
+info
+Info argument (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+baseptr
+Pointer to beginning of memory segment allocated.
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Alloc_mem allocates \fIsize\fP bytes of memory. The starting address
+of this memory is returned in the variable \fIbase\fP.
+.sp
+
+.SH FORTRAN NOTES
+.ft R
+There is no portable FORTRAN 77 syntax for using MPI_Alloc_mem.
+There is no portable Fortran syntax for using pointers returned
+from MPI_Alloc_mem. However, MPI_Alloc_mem can be used with Sun
+Fortran compilers.
+.sp
+From FORTRAN 77, you can use the following non-standard
+declarations for the SIZE and BASEPTR arguments:
+.nf
+           INCLUDE "mpif.h"
+           INTEGER*MPI_ADDRESS_KIND SIZE, BASEPTR
+.fi
+.sp
+From either FORTRAN 77 or Fortran 90, you can use "Cray pointers"
+for the BASEPTR argument. Cray pointers are described further in
+the Fortran User's Guide and are supported by many Fortran compilers.
+For example,
+.sp
+.nf
+           INCLUDE "mpif.h"
+           REAL*4 A(100,100)
+           POINTER (BASEPTR, A)
+           INTEGER*MPI_ADDRESS_KIND SIZE
+
+           SIZE = 4 * 100 * 100
+           CALL MPI_ALLOC_MEM(SIZE,MPI_INFO_NULL,BASEPTR,IERR)
+
+           ! use A
+
+           CALL MPI_FREE_MEM(A, IERR)
+.fi
+.ft R
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Free_mem

macx64/mpi/openmpi/share/man/man3/MPI_Allreduce.3

@@ -0,0 +1,138 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2007-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Allreduce 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Allreduce, MPI_Iallreduce\fP \- Combines values from all processes and distributes the result back to all processes.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Allreduce(const void \fI*sendbuf\fP, void \fI*recvbuf\fP, int\fI count\fP,
+                  MPI_Datatype\fI datatype\fP, MPI_Op\fI op\fP, MPI_Comm\fI comm\fP)
+
+int MPI_Iallreduce(const void \fI*sendbuf\fP, void \fI*recvbuf\fP, int\fI count\fP,
+                   MPI_Datatype\fI datatype\fP, MPI_Op\fI op\fP, MPI_Comm\fI comm\fP,
+                   MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+sendbuf
+Starting address of send buffer (choice).
+.TP 1i
+count
+Number of elements in send buffer (integer).
+.TP 1i
+datatype
+Datatype of elements of send buffer (handle).
+.TP 1i
+op
+Operation (handle).
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+recvbuf
+Starting address of receive buffer (choice).
+.TP 1i
+request
+Request (handle, non-blocking only).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Same as MPI_Reduce except that the result appears in the receive buffer of all the group members.
+.sp
+\fBExample 1:\fR A routine that computes the product of a vector and an array that are distributed across a group of processes and returns the answer at all nodes (compare with Example 2, with MPI_Reduce, below).
+.sp
+.nf
+SUBROUTINE PAR_BLAS2(m, n, a, b, c, comm)
+REAL a(m), b(m,n)    ! local slice of array
+REAL c(n)            ! result
+REAL sum(n)
+INTEGER n, comm, i, j, ierr
+
+! local sum
+DO j= 1, n
+  sum(j) = 0.0
+  DO i = 1, m
+    sum(j) = sum(j) + a(i)*b(i,j)
+  END DO
+END DO
+
+! global sum
+CALL MPI_ALLREDUCE(sum, c, n, MPI_REAL, MPI_SUM, comm, ierr)
+
+! return result at all nodes
+RETURN
+.fi
+.sp
+\fBExample 2:\fR A routine that computes the product of a vector and an array that are distributed across a group of processes and returns the answer at node zero.
+.sp
+.nf
+SUBROUTINE PAR_BLAS2(m, n, a, b, c, comm)
+REAL a(m), b(m,n)    ! local slice of array
+REAL c(n)            ! result
+REAL sum(n)
+INTEGER n, comm, i, j, ierr
+
+! local sum
+DO j= 1, n
+  sum(j) = 0.0
+  DO i = 1, m
+    sum(j) = sum(j) + a(i)*b(i,j)
+  END DO
+END DO
+
+! global sum
+CALL MPI_REDUCE(sum, c, n, MPI_REAL, MPI_SUM, 0, comm, ierr)
+
+! return result at node zero (and garbage at the other nodes)
+RETURN
+.fi
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-reduce operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR at all processes.
+.sp
+Note that MPI_IN_PLACE is a special kind of value; it has the same restrictions on its use as MPI_BOTTOM.
+.sp
+Because the in-place option converts the receive buffer into a send-and-receive buffer, a Fortran binding that includes INTENT must mark these as INOUT, not OUT.
+.sp
+.SH WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+When the communicator is an inter-communicator, the reduce operation occurs in two phases.  The data is reduced from all the members of the first group and received by all the members of the second group.  Then the data is reduced from all the members of the second group and received by all the members of the first.  The operation exhibits a symmetric, full-duplex behavior.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+.SH NOTES ON COLLECTIVE OPERATIONS
+
+The reduction functions (
+.I MPI_Op
+) do not return an error value.  As a result,
+if the functions detect an error, all they can do is either call
+.I MPI_Abort
+or silently skip the problem.  Thus, if you change the error handler from
+.I MPI_ERRORS_ARE_FATAL
+to something else, for example,
+.I MPI_ERRORS_RETURN
+,
+then no error may be indicated.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Alltoall.3

@@ -0,0 +1,127 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Alltoall 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+\fBMPI_Alltoall, MPI_Ialltoall\fP \- All processes send data to all processes
+
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Alltoall(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP)
+
+int MPI_Ialltoall(const void *\fIsendbuf\fP, int \fIsendcount\fP,
+	MPI_Datatype \fIsendtype\fP, void *\fIrecvbuf\fP, int \fIrecvcount\fP,
+	MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP, MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1.2i
+sendbuf
+Starting address of send buffer (choice).
+.TP 1.2i
+sendcount
+Number of elements to send to each process (integer).
+.TP 1.2i
+sendtype
+Datatype of send buffer elements (handle).
+.TP 1.2i
+recvcount
+Number of elements to receive from each process (integer).
+.TP 1.2i
+recvtype
+Datatype of receive buffer elements (handle).
+.TP 1.2i
+comm
+Communicator over which data is to be exchanged (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.2i
+recvbuf
+Starting address of receive buffer (choice).
+.TP 1.2i
+request
+Request (handle, non-blocking only).
+.ft R
+.TP 1.2i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Alltoall is a collective operation in which all processes send the same amount of data to each other, and receive the same amount of data from each other. The operation of this routine can be represented as follows, where each process performs 2n (n being the number of processes in communicator \fIcomm\fP) independent point-to-point communications (including communication with itself).
+.sp
+.nf
+	MPI_Comm_size(\fIcomm\fP, &n);
+	for (i = 0, i < n; i++)
+	    MPI_Send(\fIsendbuf\fP + i * \fIsendcount\fP * extent(\fIsendtype\fP),
+	        \fIsendcount\fP, \fIsendtype\fP, i, ..., \fIcomm\fP);
+	for (i = 0, i < n; i++)
+	    MPI_Recv(\fIrecvbuf\fP + i * \fIrecvcount\fP * extent(\fIrecvtype\fP),
+	        \fIrecvcount\fP, \fIrecvtype\fP, i, ..., \fIcomm\fP);
+.fi
+.sp
+Each process breaks up its local \fIsendbuf\fP into n blocks \- each
+containing \fIsendcount\fP elements of type \fIsendtype\fP \- and
+divides its \fIrecvbuf\fP similarly according to \fIrecvcount\fP and
+\fIrecvtype\fP. Process j sends the k-th block of its local
+\fIsendbuf\fP to process k, which places the data in the j-th block of
+its local \fIrecvbuf\fP. The amount of data sent must be equal to the
+amount of data received, pairwise, between every pair of processes.
+
+WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the gather operation occurs in two phases.  The data is gathered from all the members of the first group and received by all the members of the second group.  Then the data is gathered from all the members of the second group and received by all the members of the first.  The operation exhibits a symmetric, full-duplex behavior.
+.sp
+The first group defines the root process.  The root process uses MPI_ROOT as the value of \fIroot\fR.  All other processes in the first group use MPI_PROC_NULL as the value of \fIroot\fR.  All processes in the second group use the rank of the root process in the first group as the value of \fIroot\fR.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-to-all operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR.  In this case, \fIsendcount\fR and \fIsendtype\fR are ignored.  The input data of each process is assumed to be in the area where that process would receive its own contribution to the receive buffer.
+
+.sp
+.SH NOTES
+.sp
+All arguments on all processes are significant. The \fIcomm\fP argument,
+in particular, must describe the same communicator on all processes.
+.sp
+There are two MPI library functions that are more general than
+MPI_Alltoall. MPI_Alltoallv allows all-to-all communication to and
+from buffers that need not be contiguous; different processes may
+send and receive different amounts of data. MPI_Alltoallw expands
+MPI_Alltoallv's functionality to allow the exchange of data with
+different datatypes.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Alltoallv
+MPI_Alltoallw
+

macx64/mpi/openmpi/share/man/man3/MPI_Alltoallv.3

@@ -0,0 +1,154 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Alltoallv 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+\fBMPI_Alltoallv, MPI_Ialltoallv\fP \- All processes send different amount of data to, and receive different amount of data from, all processes
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Alltoallv(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], MPI_Datatype \fIsendtype\fP,
+	void *\fIrecvbuf\fP, const int\fI recvcounts\fP[],
+	const int \fIrdispls\fP[], MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP)
+
+int MPI_Ialltoallv(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], MPI_Datatype \fIsendtype\fP,
+	void *\fIrecvbuf\fP, const int\fI recvcounts\fP[],
+	const int \fIrdispls\fP[], MPI_Datatype \fIrecvtype\fP, MPI_Comm \fIcomm\fP,
+	MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1.2i
+sendbuf
+Starting address of send buffer.
+.TP 1.2i
+sendcounts
+Integer array, where entry i specifies the number of elements to send
+to rank i.
+.TP 1.2i
+sdispls
+Integer array, where entry i specifies the displacement (offset from
+\fIsendbuf\fP, in units of \fIsendtype\fP) from which to send data to
+rank i.
+.TP 1.2i
+sendtype
+Datatype of send buffer elements.
+.TP 1.2i
+recvcounts
+Integer array, where entry j specifies the number of elements to
+receive from rank j.
+.TP 1.2i
+rdispls
+Integer array, where entry j specifies the displacement (offset from
+\fIrecvbuf\fP, in units of \fIrecvtype\fP) to which data from rank j
+should be written.
+.TP 1.2i
+recvtype
+Datatype of receive buffer elements.
+.TP 1.2i
+comm
+Communicator over which data is to be exchanged.
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.2i
+recvbuf
+Address of receive buffer.
+.TP 1.2i
+request
+Request (handle, non-blocking only).
+.ft R
+.TP 1.2i
+IERROR
+Fortran only: Error status.
+
+.SH DESCRIPTION
+.ft R
+MPI_Alltoallv is a generalized collective operation in which all
+processes send data to and receive data from all other processes. It
+adds flexibility to MPI_Alltoall by allowing the user to specify data
+to send and receive vector-style (via a displacement and element
+count). The operation of this routine can be thought of as follows,
+where each process performs 2n (n being the number of processes in
+communicator \fIcomm\fP) independent point-to-point communications
+(including communication with itself).
+.sp
+.nf
+	MPI_Comm_size(\fIcomm\fP, &n);
+	for (i = 0, i < n; i++)
+	    MPI_Send(\fIsendbuf\fP + \fIsdispls\fP[i] * extent(\fIsendtype\fP),
+	        \fIsendcounts\fP[i], \fIsendtype\fP, i, ..., \fIcomm\fP);
+	for (i = 0, i < n; i++)
+	    MPI_Recv(\fIrecvbuf\fP + \fIrdispls\fP[i] * extent(\fIrecvtype\fP),
+	        \fIrecvcounts\fP[i], \fIrecvtype\fP, i, ..., \fIcomm\fP);
+.fi
+.sp
+Process j sends the k-th block of its local \fIsendbuf\fP to process
+k, which places the data in the j-th block of its local
+\fIrecvbuf\fP.
+.sp
+When a pair of processes exchanges data, each may pass different
+element count and datatype arguments so long as the sender specifies
+the same amount of data to send (in bytes) as the receiver expects
+to receive.
+.sp
+Note that process i may send a different amount of data to process j
+than it receives from process j. Also, a process may send entirely
+different amounts of data to different processes in the communicator.
+
+.sp
+WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the gather operation occurs in two phases.  The data is gathered from all the members of the first group and received by all the members of the second group.  Then the data is gathered from all the members of the second group and received by all the members of the first.  The operation exhibits a symmetric, full-duplex behavior.
+.sp
+The first group defines the root process.  The root process uses MPI_ROOT as the value of \fIroot\fR.  All other processes in the first group use MPI_PROC_NULL as the value of \fIroot\fR.  All processes in the second group use the rank of the root process in the first group as the value of \fIroot\fR.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+.sp
+
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-to-all operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR.  In this case, \fIsendcounts\fR, \fIsdispls\fP, and \fIsendtype\fR are ignored.  The input data of each process is assumed to be in the area where that process would receive its own contribution to the receive buffer.
+
+.SH NOTES
+.sp
+The specification of counts and displacements should not cause
+any location to be written more than once.
+.sp
+All arguments on all processes are significant. The \fIcomm\fP argument,
+in particular, must describe the same communicator on all processes.
+.sp
+The offsets of \fIsdispls\fP and \fIrdispls\fP are measured in units
+of \fIsendtype\fP and \fIrecvtype\fP, respectively. Compare this to
+MPI_Alltoallw, where these offsets are measured in bytes.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Alltoall
+MPI_Alltoallw
+

macx64/mpi/openmpi/share/man/man3/MPI_Alltoallw.3

@@ -0,0 +1,156 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Alltoallw 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+\fBMPI_Alltoallw, MPI_Ialltoallw\fP \- All processes send data of different types to, and receive data of different types from, all processes
+
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Alltoallw(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], const MPI_Datatype \fIsendtypes\fP[],
+	void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[], const int \fIrdispls\fP[],
+	const MPI_Datatype \fIrecvtypes\fP[], MPI_Comm \fIcomm\fP)
+
+int MPI_Ialltoallw(const void *\fIsendbuf\fP, const int \fIsendcounts\fP[],
+	const int \fIsdispls\fP[], const MPI_Datatype \fIsendtypes\fP[],
+	void *\fIrecvbuf\fP, const int \fIrecvcounts\fP[], const int \fIrdispls\fP[],
+	const MPI_Datatype \fIrecvtypes\fP[], MPI_Comm \fIcomm\fP,
+	MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1.2i
+sendbuf
+Starting address of send buffer.
+.TP 1.2i
+sendcounts
+Integer array, where entry i specifies the number of elements to send
+to rank i.
+.TP 1.2i
+sdispls
+Integer array, where entry i specifies the displacement (in bytes,
+offset from \fIsendbuf\fP) from which to send data to rank i.
+.TP 1.2i
+sendtypes
+Datatype array, where entry i specifies the datatype to use when
+sending data to rank i.
+.TP 1.2i
+recvcounts
+Integer array, where entry j specifies the number of elements to
+receive from rank j.
+.TP 1.2i
+rdispls
+Integer array, where entry j specifies the displacement (in bytes,
+offset from \fIrecvbuf\fP) to which data from rank j should
+be written.
+.TP 1.2i
+recvtypes
+Datatype array, where entry j specifies the datatype to use when
+receiving data from rank j.
+.TP 1.2i
+comm
+Communicator over which data is to be exchanged.
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.2i
+recvbuf
+Address of receive buffer.
+.TP 1.2i
+request
+Request (handle, non-blocking only).
+.ft R
+.TP 1.2i
+IERROR
+Fortran only: Error status.
+
+.SH DESCRIPTION
+.ft R
+MPI_Alltoallw is a generalized collective operation in which all
+processes send data to and receive data from all other processes. It
+adds flexibility to MPI_Alltoallv by allowing the user to specify the
+datatype of individual data blocks (in addition to displacement and
+element count). Its operation can be thought of in the following way,
+where each process performs 2n (n being the number of processes in
+communicator \fIcomm\fP) independent point-to-point communications
+(including communication with itself).
+.sp
+.nf
+	MPI_Comm_size(\fIcomm\fP, &n);
+	for (i = 0, i < n; i++)
+	    MPI_Send(\fIsendbuf\fP + \fIsdispls\fP[i], \fIsendcounts\fP[i],
+	        \fIsendtypes\fP[i], i, ..., \fIcomm\fP);
+	for (i = 0, i < n; i++)
+	    MPI_Recv(\fIrecvbuf\fP + \fIrdispls\fP[i], \fIrecvcounts\fP[i],
+	        \fIrecvtypes\fP[i], i, ..., \fIcomm\fP);
+.fi
+.sp
+Process j sends the k-th block of its local \fIsendbuf\fP to process
+k, which places the data in the j-th block of its local
+\fIrecvbuf\fP.
+.sp
+When a pair of processes exchanges data, each may pass different
+element count and datatype arguments so long as the sender specifies
+the same amount of data to send (in bytes) as the receiver expects
+to receive.
+.sp
+Note that process i may send a different amount of data to process j
+than it receives from process j. Also, a process may send entirely
+different amounts and types of data to different processes in the
+communicator.
+
+WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the gather operation occurs in two phases.  The data is gathered from all the members of the first group and received by all the members of the second group.  Then the data is gathered from all the members of the second group and received by all the members of the first.  The operation exhibits a symmetric, full-duplex behavior.
+.sp
+The first group defines the root process.  The root process uses MPI_ROOT as the value of \fIroot\fR.  All other processes in the first group use MPI_PROC_NULL as the value of \fIroot\fR.  All processes in the second group use the rank of the root process in the first group as the value of \fIroot\fR.
+.sp
+When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase.
+.sp
+
+.SH USE OF IN-PLACE OPTION
+When the communicator is an intracommunicator, you can perform an all-to-all operation in-place (the output buffer is used as the input buffer).  Use the variable MPI_IN_PLACE as the value of \fIsendbuf\fR.  In this case, \fIsendcounts\fR, \fIsdispls\fP, and \fIsendtypes\fR are ignored.  The input data of each process is assumed to be in the area where that process would receive its own contribution to the receive buffer.
+
+.SH NOTES
+.sp
+The specification of counts, types, and displacements should not cause
+any location to be written more than once.
+.sp
+All arguments on all processes are significant. The \fIcomm\fP argument,
+in particular, must describe the same communicator on all processes.
+.sp
+The offsets of \fIsdispls\fP and \fIrdispls\fP are measured in bytes.
+Compare this to MPI_Alltoallv, where these offsets are measured in units
+of \fIsendtype\fP and \fIrecvtype\fP, respectively.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Alltoall
+MPI_Alltoallv
+

macx64/mpi/openmpi/share/man/man3/MPI_Attr_delete.3

@@ -0,0 +1,61 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Attr_delete 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Attr_delete\fP \- Deletes attribute value associated with a key -- use of this routine is deprecated.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Attr_delete(MPI_Comm\fI comm\fP, int\fI keyval\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator to which attribute is attached (handle).
+.TP 1i
+keyval
+The key value of the deleted attribute (integer).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+Note that use of this routine is \fIdeprecated\fP as of MPI-2, and
+was \fIdeleted\fP in MPI-3. Please use MPI_Comm_delete_attr.  This
+function does not have a C++ or mpi_f08 binding.
+.sp
+Delete attribute from cache by key. This function invokes the attribute delete function delete_fn specified when the keyval was created. The call will fail if the delete_fn function returns an error code other than MPI_SUCCESS.
+
+Whenever a communicator is replicated using the function MPI_Comm_dup, all callback copy functions for attributes that are currently set are invoked (in arbitrary order). Whenever a communicator is deleted using the function MPI_Comm_free, all callback delete functions for attributes that are currently set are invoked.
+
+
+.SH NOTES
+Note that it is not defined by the MPI standard what happens if the
+delete_fn callback invokes other MPI functions.  In Open MPI, it is
+not valid for delete_fn callbacks (or any of their children) to add or
+delete attributes on the same object on which the delete_fn callback
+is being invoked.
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+MPI_Comm_delete_attr
+.br
+

macx64/mpi/openmpi/share/man/man3/MPI_Attr_get.3

@@ -0,0 +1,61 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Attr_get 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Attr_get\fP \- Retrieves attribute value by key -- use of this routine is deprecated.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Attr_get(MPI_Comm \fIcomm\fP, int\fI keyval\fP,void\fI *attribute_val\fP,
+	int\fI *flag \fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator to which attribute is attached (handle).
+.TP 1i
+keyval
+ Key value (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+attribute_val
+Attribute value, unless flag = false.
+.TP 1i
+flag
+True if an attribute value was extracted; false if no attribute is associated with the key.
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Note that use of this routine is \fIdeprecated\fP as of MPI-2, and
+was \fIdeleted\fP in MPI-3. Please use MPI_Comm_get_attr.  This
+function does not have a C++ or mpi_f08 binding.
+.sp
+Retrieves attribute value by key. The call is erroneous if there is no key
+with value keyval. On the other hand, the call is correct if the key value exists, but no attribute is attached on comm for that key; in such case, the call returns flag = false. In particular MPI_KEYVAL_INVALID is an erroneous key value.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+
+.SH SEE ALSO
+
+MPI_Comm_get_attr
+.br

macx64/mpi/openmpi/share/man/man3/MPI_Attr_put.3

@@ -0,0 +1,64 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Attr_put 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Attr_put\fP \- Stores attribute value associated with a key -- use of this routine is deprecated.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Attr_put(MPI_Comm \fIcomm\fP, int\fI keyval\fP, void\fI *attribute_val\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator to which attribute will be attached (handle).
+.TP 1i
+keyval
+Key value, as returned by MPI_KEYVAL_CREATE (integer).
+.TP 1i
+attribute_val
+Attribute value.
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Note that use of this routine is \fIdeprecated\fP as of MPI-2, and
+was \fIdeleted\fP in MPI-3. Please use MPI_Comm_set_attr.  This
+function does not have a C++ or mpi_f08 binding.
+.sp
+MPI_Attr_put stores the stipulated attribute value attribute_val for subsequent retrieval by MPI_Attr_get. If the value is already present, then the outcome is as if MPI_Attr_delete was first called to delete the previous value (and the callback function delete_fn was executed), and a new value was next stored. The call is erroneous if there is no key with value keyval; in particular MPI_KEYVAL_INVALID is an erroneous key value. The call will fail if the delete_fn function returned an error code other than MPI_SUCCESS.
+
+.SH NOTES
+.ft R
+Values of the permanent attributes MPI_TAG_UB, MPI_HOST,
+MPI_IO, and MPI_WTIME_IS_GLOBAL may not be changed.
+.sp
+The type of the attribute value depends on whether C or Fortran is being used. In C, an attribute value is a pointer (void *); in Fortran, it is a single integer (not a pointer, since Fortran has no pointers and there are systems for which a pointer does not fit in an integer, e.g., any  32-bit address system that uses 64 bits for Fortran DOUBLE PRECISION).
+.sp
+If an attribute is already present, the delete function (specified when the corresponding keyval was created) will be called.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Comm_set_attr
+.br

macx64/mpi/openmpi/share/man/man3/MPI_Barrier.3

@@ -0,0 +1,52 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2014-2015 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Barrier 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Barrier, MPI_Ibarrier\fP \- Synchronization between MPI processes in a group
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Barrier(MPI_Comm \fIcomm\fP)
+
+int MPI_Ibarrier(MPI_Comm \fIcomm\fP, MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+request
+Request (handle, non-blocking only).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+An MPI barrier completes after all group members have entered the
+barrier.
+
+.SH WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the barrier operation is performed across all processes in both groups.  All processes in the first group may exit the barrier when all processes in the second group have entered the barrier.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+MPI_Bcast

macx64/mpi/openmpi/share/man/man3/MPI_Bcast.3

@@ -0,0 +1,81 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Bcast 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Bcast, MPI_Ibcast\fP \- Broadcasts a message from the process with rank \fIroot\fP to all other processes of the group.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Bcast(void \fI*buffer\fP, int\fI count\fP, MPI_Datatype\fI datatype\fP,
+	int\fI root\fP, MPI_Comm\fI comm\fP)
+
+int MPI_Ibcast(void \fI*buffer\fP, int\fI count\fP, MPI_Datatype\fI datatype\fP,
+	int\fI root\fP, MPI_Comm\fI comm\fP, MPI_Request \fI*request\fP)
+
+.fi
+.SH INPUT/OUTPUT PARAMETERS
+.ft R
+.TP 1i
+buffer
+Starting address of buffer (choice).
+.TP 1i
+count
+Number of entries in buffer (integer).
+.TP 1i
+datatype
+Data type of buffer (handle).
+.TP 1i
+root
+Rank of broadcast root (integer).
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+request
+Request (handle, non-blocking only).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Bcast broadcasts a message from the process with rank root to all processes of the group, itself included. It is called by all members of group using the same arguments for comm, root. On return, the contents of root's communication buffer has been copied to all processes.
+.sp
+General, derived datatypes are allowed for datatype. The type signature of count, datatype on any process must be equal to the type signature of count, datatype at the root. This implies that the amount of data sent must be equal to the amount received, pairwise between each process and the root. MPI_Bcast and all other data-movement collective routines make this restriction. Distinct type maps between sender and receiver are still allowed.
+.sp
+\fBExample:\fR Broadcast 100 ints from process 0 to every process in the group.
+.nf
+    MPI_Comm comm;
+    int array[100];
+    int root=0;
+    \&...
+    MPI_Bcast( array, 100, MPI_INT, root, comm);
+.fi
+.sp
+As in many of our sample code fragments, we assume that some of the variables (such as comm in the example above) have been assigned appropriate values.
+.sp
+.SH WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
+.sp
+When the communicator is an inter-communicator, the root process in the first group broadcasts data to all the processes in the second group.  The first group defines the root process.  That process uses MPI_ROOT as the value of its \fIroot\fR argument.  The remaining processes use MPI_PROC_NULL as the value of their \fIroot\fR argument.  All processes in the second group use the rank of that root process in the first group as the value of their \fIroot\fR argument.   The receive buffer arguments of the processes in the second group must be consistent with the send buffer argument of the root process in the first group.
+.sp
+.SH NOTES
+This function does not support the in-place option.
+.sp
+
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Bsend.3

@@ -0,0 +1,96 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Bsend 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Bsend\fP \- Basic send with user-specified buffering.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Bsend(const void \fI*buf\fP, int\fI count\fP, MPI_Datatype\fI datatype\fP,
+	int\fI dest\fP, int\fI tag\fP, MPI_Comm\fI comm\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+buf
+Initial address of send buffer (choice).
+.TP 1i
+count
+Number of entries in send buffer (nonnegative integer).
+.TP 1i
+datatype
+Datatype of each send buffer element (handle).
+.TP 1i
+dest
+Rank of destination (integer).
+.TP 1i
+tag
+Message tag (integer).
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Bsend performs a buffered-mode, blocking send.
+
+.SH NOTES
+.ft R
+This send is provided as a convenience function; it allows the user to send messages without worrying about where they are buffered (because the user must have provided buffer space with MPI_Buffer_attach).
+.sp
+In deciding how much buffer space to allocate, remember that the buffer space
+is not available for reuse by subsequent \fIMPI_Bsend\fPs unless you are certain
+that the message
+has been received (not just that it should have been received).  For example,
+this code does not allocate enough buffer space:
+.nf
+
+    MPI_Buffer_attach( b, n*sizeof(double) + MPI_BSEND_OVERHEAD );
+    for (i=0; i<m; i++) {
+        MPI_Bsend( buf, n, MPI_DOUBLE, ... );
+    }
+
+.fi
+because only enough buffer space is provided for a single send, and the
+loop may start a second
+.I MPI_Bsend
+before the first is done making use of the
+buffer.
+
+In C, you can
+force the messages to be delivered by
+MPI_Buffer_detach( &b, &n );
+MPI_Buffer_attach( b, n );
+(The
+.I MPI_Buffer_detach
+will not complete until all buffered messages are
+delivered.)
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Buffer_attach
+MPI_Ibsend
+MPI_Bsend_init
+

macx64/mpi/openmpi/share/man/man3/MPI_Bsend_init.3

@@ -0,0 +1,72 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Bsend_init 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Bsend_init\fP \- Builds a handle for a buffered send.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Bsend_init(const void \fI*buf\fP, int\fI count\fP, MPI_Datatype\fI datatype\fP,
+	int\fI dest\fP, int\fI tag\fP, MPI_Comm\fI comm\fP, MPI_Request\fI *request\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+buf
+Initial address of send buffer (choice).
+.TP 1i
+count
+Number of elements sent (integer).
+.TP 1i
+datatype
+Type of each element (handle).
+.TP 1i
+dest
+Rank of destination (integer).
+.TP 1i
+tag
+Message tag (integer).
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+request
+Communication request (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Creates a persistent communication request for a buffered mode send, and binds to it all the arguments of a send operation.
+.sp
+A communication (send or receive) that uses a persistent request is initiated by the function MPI_Start.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Send_init
+MPI_Ssend_init
+MPI_Rsend_init
+MPI_Recv_init
+MPI_Start
+MPI_Startall
+

macx64/mpi/openmpi/share/man/man3/MPI_Buffer_attach.3

@@ -0,0 +1,68 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Buffer_attach 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Buffer_attach\fP \- Attaches a user-defined buffer for sending.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Buffer_attach(void \fI*buf\fP, int\fI size\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+buf
+Initial buffer address (choice).
+.TP 1i
+size
+Buffer size, in bytes (integer).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Provides to MPI a buffer in the user's memory to be used for buffering outgoing messages. The buffer is used only by messages sent in buffered mode. Only one buffer can be attached to a process at a time.
+
+.SH NOTES
+.ft R
+The size given should be the sum of the sizes of all outstanding Bsends that you intend to have, plus MPI_BSEND_OVERHEAD bytes for each Bsend that you do. For the purposes of calculating size, you should use MPI_Pack_size. In other words, in the code
+.sp
+.nf
+    MPI_Buffer_attach( buf, size );
+    MPI_Bsend( \&..., count=20, datatype=type1, \&... );
+    \&...
+    MPI_Bsend( \&..., count=40, datatype=type2, \&... );
+.fi
+.sp
+the value of size in the MPI_Buffer_attach call should be greater than the value computed by
+.sp
+.nf
+    MPI_Pack_size( 20, type1, comm, &s1 );
+    MPI_Pack_size( 40, type2, comm, &s2 );
+    size = s1 + s2 + 2 * MPI_BSEND_OVERHEAD;
+.fi
+.sp
+MPI_BSEND_OVERHEAD gives the maximum amount of buffer space that may be used by the Bsend routines. This value is in mpi.h for C and mpif.h for Fortran.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Buffer_detach
+

macx64/mpi/openmpi/share/man/man3/MPI_Buffer_detach.3

@@ -0,0 +1,85 @@
+.\" -*- nroff -*-
+.\" Copyright 2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Buffer_detach 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Buffer_detach\fP \- Removes an existing buffer (for use in MPI_Bsend, etc.)
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Buffer_detach(void \fI*buf\fP, int\fI *size\fP)
+
+.fi
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+buf
+Initial buffer address (choice).
+.TP 1i
+size
+Buffer size, in bytes (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+Detach the buffer currently associated with MPI. The call returns the address and the size of the detached buffer. This operation will block until all messages currently in the buffer have been transmitted. Upon return of this function, the user may reuse or deallocate the space taken by the buffer.
+.sp
+\fBExample:\fP Calls to attach and detach buffers.
+.sp
+.nf
+    #define BUFFSIZE 10000
+    int size
+    char *buff;
+    MPI_Buffer_attach( malloc(BUFFSIZE), BUFFSIZE);
+    /* a buffer of 10000 bytes can now be used by MPI_Bsend */
+    MPI_Buffer_detach( &buff, &size);
+    /* Buffer size reduced to zero */
+    MPI_Buffer_attach( buff, size);
+    /* Buffer of 10000 bytes available again */
+.fi
+
+.SH NOTES
+.ft R
+The reason that MPI_Buffer_detach returns the address and size of the buffer being detached is to allow nested libraries to replace and restore the buffer. For example, consider
+.sp
+.nf
+    int size, mysize, idummy;
+    void *ptr, *myptr, *dummy;
+    MPI_Buffer_detach( &ptr, &size );
+    MPI_Buffer_attach( myptr, mysize );
+    \&...
+    \&... library code \&...
+    \&...
+    MPI_Buffer_detach( &dummy, &idummy );
+    MPI_Buffer_attach( ptr, size );
+.fi
+.sp
+This is much like the action of the UNIX signal routine and has the same strengths (it's simple) and weaknesses (it only works for nested usages).
+.sp
+\fBFor Fortran:\fP The Fortran binding for this routine is different. Because Fortran does not have pointers, it is impossible to provide a way to use the output of this routine to exchange buffers. In this case, only the size field is set.
+.sp
+\fBFor C:\fP Even though the buf argument is declared as void, it is really the address of a void pointer. See Rationale, below, for more details.
+.sp
+Even though the C functions MPI_Buffer_attach and
+MPI_Buffer_detach both have a first argument of type void*, these arguments are used differently: A pointer to the buffer is passed to MPI_Buffer_attach; the address of the pointer is passed to MPI_Buffer_detach, so that this call can return the pointer value.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Buffer_attach
+MPI_Bsend

macx64/mpi/openmpi/share/man/man3/MPI_Cancel.3

@@ -0,0 +1,67 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cancel 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cancel\fP \- Cancels a communication request.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cancel(MPI_Request\fI *request\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+request
+Communication request (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+The MPI_Cancel operation allows pending communications to be canceled. This is required for cleanup. Posting a send or a receive ties up user resources (send or receive buffers), and a cancel may be needed to free these resources gracefully.
+.sp
+A call to MPI_Cancel marks for cancellation a pending, nonblocking communication operation (send or receive). The cancel call is local. It returns immediately, possibly before the communication is actually canceled. It is still necessary to complete a communication that has been marked for cancellation, using a call to MPI_Request_free, MPI_Wait, or MPI_Test (or any of the derived operations).
+.sp
+If a communication is marked for cancellation, then an MPI_Wait call for that communication is guaranteed to return, irrespective of the activities of other processes (i.e., MPI_Wait behaves as a local function); similarly if MPI_Test is repeatedly called in a busy wait loop for a canceled communication, then MPI_Test will eventually be successful.
+.sp
+MPI_Cancel can be used to cancel a communication that uses a persistent request (see Section 3.9 in the MPI-1 Standard, "Persistent Communication Requests") in the same way it is used for nonpersistent requests. A successful cancellation cancels the active communication, but not the request itself. After the call to MPI_Cancel and the subsequent call to MPI_Wait or MPI_Test, the request becomes inactive and can be activated for a new communication.
+.sp
+The successful cancellation of a buffered send frees the buffer space occupied by the pending message.
+.sp
+Either the cancellation succeeds or the communication succeeds, but not both. If a send is marked for cancellation, then it must be the case that either the send completes normally, in which case the message sent is received at the destination process, or that the send is successfully canceled, in which case no part of the message is received at the destination. Then, any matching receive has to be satisfied by another send. If a receive is marked for cancellation, then it must be the case that either the receive completes normally, or that the receive is successfully canceled, in which case no part of the receive buffer is altered. Then, any matching send has to be satisfied by another receive.
+.sp
+If the operation has been canceled, then information to that effect will be returned in the status argument of the operation that completes the communication.
+
+.SH NOTES
+.ft R
+The primary expected use of MPI_Cancel is in multi-buffering schemes,
+where speculative MPI_Irecvs are made.  When the
+computation completes, some of these requests may remain;
+using MPI_Cancel allows the user to cancel these unsatisfied requests.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Probe
+MPI_Iprobe
+MPI_Test_cancelled
+MPI_Cart_coords
+

macx64/mpi/openmpi/share/man/man3/MPI_Cart_coords.3

@@ -0,0 +1,50 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_coords 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_coords\fP \- Determines process coords in Cartesian topology given rank in group.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_coords(MPI_Comm \fIcomm\fP, int\fI rank\fP, int\fI maxdims\fP,
+	int\fI coords\fP[])
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+.TP 1i
+rank
+Rank of a process within group of comm (integer).
+.TP 1i
+maxdims
+ Length of vector coords in the calling program (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+coords
+Integer array (of size ndims,which was defined by MPI_Cart_create call) containing the Cartesian coordinates of specified process (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Cart_coords provies a mapping of ranks to Cartesian coordinates.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Cart_create.3

@@ -0,0 +1,59 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_create 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_create\fP \- Makes a new communicator to which Cartesian topology information has been attached.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_create(MPI_Comm\fI comm_old\fP, int\fI ndims\fP, const int\fI dims\fP[],
+	const int\fI periods\fP[], int\fI reorder\fP, MPI_Comm\fI *comm_cart\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm_old
+Input communicator (handle).
+.TP 1i
+ndims
+Number of dimensions of Cartesian grid (integer).
+.TP 1i
+dims
+Integer array of size ndims specifying the number of processes in each
+dimension.
+.TP 1i
+periods
+Logical array of size ndims specifying whether the grid is periodic (true)
+or not (false) in each dimension.
+.TP 1i
+reorder
+Ranking may be reordered (true) or not (false) (logical).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+comm_cart
+Communicator with new Cartesian topology (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Cart_create returns a handle to a new communicator to which the Cartesian topology information is attached. If reorder = false then the rank of each process in the new group is identical to its rank in the old group. Otherwise, the function may reorder the processes (possibly so as to choose a good embedding of the virtual topology onto the physical machine). If the total size of the Cartesian grid is smaller than the size of the group of comm, then some processes are returned MPI_COMM_NULL, in analogy to MPI_Comm_split. The call is erroneous if it specifies a grid that is larger than the group size.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Cart_get.3

@@ -0,0 +1,59 @@
+.\" -*- nroff -*-
+.\" Copyright 2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_get 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_get\fP \-  Retrieves Cartesian topology information associated with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_get(MPI_Comm\fI comm\fP, int\fI maxdims\fP, int\fI dims\fP[], int\fI periods\fP[],
+	int\fI coords\fP[])
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+.TP 1i
+maxdims
+Length of vectors dims, periods, and coords in the calling program (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+dims
+Number of processes for each Cartesian dimension (array of integers).
+.TP 1i
+periods
+Periodicity (true/false) for each Cartesian dimension (array of logicals).
+.TP 1i
+coords
+Coordinates of calling process in Cartesian structure (array of integers).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+The functions MPI_Cartdim_get and MPI_Cart_get return the Cartesian topology information that was associated with a communicator by MPI_Cart_create.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Cartdim_get
+MPI_Cart_create

macx64/mpi/openmpi/share/man/man3/MPI_Cart_map.3

@@ -0,0 +1,61 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_map 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_map \fP \-  Maps process to Cartesian topology information.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_map(MPI_Comm \fIcomm\fP, int\fI ndims\fP, const int\fI dims\fP[],
+	const int\fI periods\fP[], int\fI *newrank\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Input communicator (handle).
+.TP 1i
+ndims
+Number of dimensions of Cartesian structure (integer).
+.TP 1i
+dims
+Integer array of size ndims specifying the number of processes in each
+coordinate direction.
+.TP 1i
+periods
+Logical array of size ndims specifying the periodicity specification in each coordinate direction.
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newrank
+Reordered rank of the calling process; MPI_UNDEFINED if calling process does not belong to grid (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Cart_map and MPI_Graph_map can be used to implement all other topology functions. In general they will not be called by the user directly, unless he or she is creating additional virtual topology capability other than that provided by MPI.
+.sp
+MPI_Cart_map computes an "optimal" placement for the calling process on the physical machine. A possible implementation of this function is to always return the rank of the calling process, that is, not to perform any reordering.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Graph_map

macx64/mpi/openmpi/share/man/man3/MPI_Cart_rank.3

@@ -0,0 +1,50 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_rank 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_rank \fP \-  Determines process rank in communicator given Cartesian location.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_rank(MPI_Comm \fIcomm\fP, int\fI coords\fP[], int\fI *rank\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+.TP 1i
+coords
+Integer array (of size ndims, which was defined by MPI_Cart_create call) specifying the Cartesian coordinates of a process.
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+rank
+Rank of specified process (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+For a process group with Cartesian structure, the function MPI_Cart_rank
+translates the logical process coordinates to process ranks as they are used by the point-to-point routines.  For dimension i with periods(i) = true, if the coordinate, coords(i), is out of range, that is, coords(i) < 0 or  coords(i) >= dims(i), it is shifted back to the interval  0 =< coords(i) < dims(i) automatically. Out-of-range coordinates are erroneous for nonperiodic dimensions.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.sp
+MPI_Cart_create

macx64/mpi/openmpi/share/man/man3/MPI_Cart_shift.3

@@ -0,0 +1,80 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Cart_shift 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_shift \fP \-  Returns the shifted source and destination ranks, given a shift direction and amount.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_shift(MPI_Comm \fIcomm\fP, int\fI direction\fP, int\fI disp\fP,
+	int\fI *rank_source\fP, int\fI *rank_dest\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+.TP 1i
+direction
+Coordinate dimension of shift (integer).
+.TP 1i
+disp
+Displacement ( > 0: upward shift, < 0: downward shift) (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+rank_source
+Rank of source process (integer).
+.TP 1i
+rank_dest
+Rank of destination process (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+If the process topology is a Cartesian structure, an MPI_Sendrecv operation is likely to be used along a coordinate direction to perform a shift of data. As input, MPI_Sendrecv takes the rank of a source process for the receive, and the rank of a destination process for the send. If the function MPI_Cart_shift is called for a Cartesian process group, it provides the calling process with the above identifiers, which then can be passed to MPI_Sendrecv. The user specifies the coordinate direction and the size of the step (positive or negative). The function is local.
+.sp
+The direction argument indicates the dimension of the shift, i.e., the coordinate whose value is modified by the shift. The coordinates are numbered from 0 to ndims-1, where ndims is the number of dimensions.
+.sp
+\fBNote:\fP  The direction argument is in the range [0, n-1] for an n-dimensional Cartesian mesh.
+.sp
+Depending on the periodicity of the Cartesian group in the specified coordinate direction, MPI_Cart_shift provides the identifiers for a circular or an end-off shift. In the case of an end-off shift, the value MPI_PROC_NULL may be returned in rank_source or rank_dest, indicating that the source or the destination for the shift is out of range.
+.sp
+\fBExample:\fP The communicator, comm, has a two-dimensional, periodic, Cartesian  topology associated with it. A two-dimensional array of REALs is stored one element per process, in variable A. One wishes to skew this array, by shifting column i (vertically, i.e., along the column) by i steps.
+.sp
+.nf
+  \&....
+  C find process rank
+        CALL MPI_COMM_RANK(comm, rank, ierr)
+  C find Cartesian coordinates
+        CALL MPI_CART_COORDS(comm, rank, maxdims, coords,
+                             ierr)
+  C compute shift source and destination
+        CALL MPI_CART_SHIFT(comm, 0, coords(2), source,
+                            dest, ierr)
+  C skew array
+        CALL MPI_SENDRECV_REPLACE(A, 1, MPI_REAL, dest, 0,
+                                  source, 0, comm, status,
+                                  ierr)
+.fi
+
+.SH NOTE
+In Fortran, the dimension indicated by DIRECTION = i has DIMS(i+1) nodes, where DIMS is the array that was used to create the grid. In C, the dimension indicated by direction = i is the dimension specified by dims[i].
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Cart_sub.3

@@ -0,0 +1,62 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cart_sub 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cart_sub \fP \- Partitions a communicator into subgroups, which form lower-dimensional Cartesian subgrids.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cart_sub(MPI_Comm \fIcomm\fP, const int\fI remain_dims\fP[], MPI_Comm\fI *comm_new\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+.TP 1i
+remain_dims
+The ith entry of remain_dims specifies whether the ith dimension is kept in the subgrid (true) or is dropped (false) (logical vector).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+comm_new
+Communicator containing the subgrid that includes the calling process (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+If a Cartesian topology has been created with MPI_Cart_create, the function  MPI_Cart_sub can be used to partition the communicator group into subgroups that form lower-dimensional Cartesian subgrids, and to build for each subgroup a communicator with the associated subgrid Cartesian topology. (This function is closely related to MPI_Comm_split.)
+.sp
+\fBExample:\fP Assume that  MPI_Cart_create( \&..., comm) has defined a (2 x 3 x 4)   grid. Let  remain_dims = (true, false, true). Then a call to
+.sp
+.nf
+    MPI_Cart_sub(comm, remain_dims, comm_new)
+.fi
+.sp
+will create three communicators, each with eight processes in a 2 x 4 Cartesian topology. If remain_dims = (false, false, true) then the call to MPI_Cart_sub(comm, remain_dims, comm_new) will create six nonoverlapping communicators, each with four processes, in a one-dimensional Cartesian topology.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Cart_create
+MPI_Comm_split
+

macx64/mpi/openmpi/share/man/man3/MPI_Cartdim_get.3

@@ -0,0 +1,50 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Cartdim_get 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Cartdim_get \fP \-  Retrieves Cartesian topology information associated with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Cartdim_get(MPI_Comm\fI comm\fP, int\fI *ndims\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator with Cartesian structure (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+ndims
+Number of dimensions of the Cartesian structure (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Cartdim_get returns the number of dimensions of the Cartesian structure.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Cart_get
+MPI_Cart_create
+

macx64/mpi/openmpi/share/man/man3/MPI_Close_port.3

@@ -0,0 +1,40 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Close_port 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Close_port \fP \- Releases the specified network address.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Close_port(const char *\fIport_name\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+port_name
+A port (string).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Close_port releases the network address represented by \fIport_name\fP.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_accept.3

@@ -0,0 +1,62 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2009-2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2007, Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_accept 3OpenMPI "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_accept \fP \- Establishes communication with a client.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_accept(const char *\fIport_name\fP, MPI_Info \fIinfo\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP, MPI_Comm *\fInewcomm\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+port_name
+Port name (string, used only on \fIroot\fP).
+.TP 1i
+info
+Options given by root for the accept (handle, used only on root). No options currently supported.
+.TP 1i
+root
+Rank in \fIcomm\fP of root node (integer).
+.TP 1i
+comm
+Intracommunicator over which call is collective (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+Intercommunicator with client as remote group (handle)
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_accept establishes communication with a client. It is collective over the calling communicator. It returns an intercommunicator that allows communication with the client, after the client has connected with the MPI_Comm_accept function using the MPI_Comm_connect function.
+.sp
+The \fIport_name\fP must have been established through a call to MPI_Open_port on the root.
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+.SH SEE ALSO
+MPI_Comm_connect
+MPI_Open_port
+.br
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_c2f.3

@@ -0,0 +1 @@
+.so man3/MPI_Comm_f2c.3

macx64/mpi/openmpi/share/man/man3/MPI_Comm_call_errhandler.3

@@ -0,0 +1,66 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_call_errhandler 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+
+\fBMPI_Comm_call_errhandler\fP \- Passes the supplied error code to the
+error handler assigned to a communicator
+
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_call_errhandler(MPI_Comm \fIcomm\fP, int \fIerrorcode\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1.4i
+comm
+communicator with error handler (handle).
+.ft R
+.TP 1.4i
+errorcode
+error code (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1.4i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function invokes the error handler assigned to the communicator
+\fIcomm\fP with the supplied error code \fIerrorcode\fP. If the error
+handler was successfully called, the process is not aborted, and the
+error handler returns, this function returns MPI_SUCCESS.
+
+.SH NOTES
+.ft R
+Users should note that the default error handler is
+MPI_ERRORS_ARE_FATAL. Thus, calling this function will abort the
+processes in \fIcomm\fP if the default error handler has not been
+changed.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Comm_create_errhandler
+MPI_Comm_set_errhandler
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_compare.3

@@ -0,0 +1,47 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_compare 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_compare \fP \-  Compares two communicators.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_compare(MPI_Comm \fIcomm1\fP, MPI_Comm\fI comm2\fP, int\fI *result\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm1
+Comm1 (handle).
+.TP 1i
+comm2
+Comm2 (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+result
+Result of comparison (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_IDENT results if and only if comm1 and comm2 are handles for the same object (identical groups and same contexts). MPI_CONGRUENT results if the underlying groups are identical in constituents and rank order; these communicators differ only by context. MPI_SIMILAR results of the group members of both communicators are the same but the rank order differs. MPI_UNEQUAL results otherwise.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_connect.3

@@ -0,0 +1,67 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2007-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_connect 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_connect \fP \- Establishes communication with a server.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_connect(const char *\fIport_name\fP, MPI_Info \fIinfo\fP, int \fIroot\fP,
+	MPI_Comm \fIcomm\fP, MPI_Comm *\fInewcomm\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+port_name
+Port name (string, used only on \fIroot\fP).
+.TP 1i
+info
+Options given by root for the connect (handle, used only on root). No options currently supported.
+.TP 1i
+root
+Rank in \fIcomm\fP of root node (integer).
+.TP 1i
+comm
+Intracommunicator over which call is collective (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+Intercommunicator with client as remote group (handle)
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_connect establishes communication with a server specified by \fIport_name\fP. It is collective over the calling communicator and returns an intercommunicator in which the remote group participated in an MPI_Comm_accept. The MPI_Comm_connect call must only be called after the MPI_Comm_accept call has been made by the MPI job acting as the server.
+.sp
+If the named port does not exist (or has been closed), MPI_Comm_connect raises an error of class MPI_ERR_PORT.
+.sp
+MPI provides no guarantee of fairness in servicing connection attempts. That is, connection attempts are not necessarily satisfied in the order in which they were initiated, and competition from other connection attempts may prevent a particular connection attempt from being satisfied.
+
+The \fIport_name\fP parameter is the address of the server. It must be the same as the name returned by MPI_Open_port on the server.
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+.SH SEE ALSO
+MPI_Comm_accept
+MPI_Open_port
+.br
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_create.3

@@ -0,0 +1,71 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_create 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_create\fP \- Creates a new communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_create(MPI_Comm \fIcomm\fP, MPI_Group\fI group\fP, MPI_Comm\fI *newcomm\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+.TP 1i
+group
+Group, which is a subset of the group of comm (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+New communicator (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function creates a new communicator newcomm with communication
+group defined by group and a new context. The function sets
+\fInewcomm\fR to a new communicator that spans all the processes that
+are in the group.  It sets \fInewcomm\fR to MPI_COMM_NULL for
+processes that are not in the group.
+
+Each process must call with a \fIgroup\fR argument that is a subgroup
+of the group associated with \fIcomm\fR; this could be
+MPI_GROUP_EMPTY. The processes may specify different values for the
+\fIgroup\fR argument. If a process calls with a non-empty \fIgroup\fR,
+then all processes in that group must call the function with the same
+\fIgroup\fR as argument, that is: the same processes in the same
+order. Otherwise the call is erroneous.
+.sp
+.LP
+
+.SH NOTES
+MPI_Comm_create provides a means of making a subset of processes for the purpose of separate MIMD computation, with separate communication space. \fInewcomm\fR, which is created by MPI_Comm_create, can be used in subsequent calls to MPI_Comm_create (or other communicator constructors) to further subdivide a computation into parallel sub-computations. A more general service is provided by MPI_Comm_split.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Comm_split
+.sp
+MPI_Intercomm_create
+MPI_Comm_create_group

macx64/mpi/openmpi/share/man/man3/MPI_Comm_create_errhandler.3

@@ -0,0 +1,76 @@
+.\" -*- nroff -*-
+.\" Copyright 2009-2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_create_errhandler 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_create_errhandler \fP \- Creates an error handler that can be attached to communicators.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_create_errhandler(MPI_Comm_errhandler_function \fI*function\fP,
+	MPI_Errhandler *\fIerrhandler\fP)
+
+.fi
+.SH DEPRECATED TYPE NAME NOTE
+.ft R
+MPI-2.2 deprecated the MPI_Comm_errhandler_fn and
+MPI::Comm::Errhandler_fn types in favor of
+MPI_Comm_errhandler_function and MPI::Comm::Errhandler_function,
+respectively.  Open MPI supports both names (indeed, the _fn names are
+typedefs to the _function names).
+
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+function
+User-defined error handling procedure (function).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+errhandler
+MPI error handler (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_create_errhandler creates an error handler that can be attached to communicators. This function is identical to MPI_Errhandler_create, the use of which is deprecated.
+.sp
+In C, the user routine should be a function of type MPI_Comm_errhandler_function, which is defined as
+.sp
+.nf
+    typedef void MPI_Comm_errhandler_function(MPI_Comm *, int *, \&...);
+.fi
+.sp
+The first argument is the communicator in use. The second is the error code
+to be returned by the MPI routine that raised the error. This typedef replaces MPI_Handler_function, the use of which is deprecated.
+.sp
+In Fortran, the user routine should be of this form:
+.sp
+.nf
+    SUBROUTINE COMM_ERRHANDLER_FUNCTION(COMM, ERROR_CODE, \&...)
+        INTEGER COMM, ERROR_CODE
+.fi
+.sp
+In C++, the user routine should be of this form:
+.sp
+.nf
+    typedef void MPI::Comm::Errhandler_function(MPI_Comm &, int *, \&...);
+.fi
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_create_group.3

@@ -0,0 +1,75 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_create_group 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_create_group\fP \- Creates a new communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_create_group(MPI_Comm \fIcomm\fP, MPI_Group\fI group\fP, int\fI tag\fP, MPI_Comm\fI *newcomm\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+.TP 1i
+group
+Group, which is a subset of the group of comm (handle).
+.TP 1i
+tag
+Tag (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+New communicator (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_create_group is similar to MPI_Comm_create; however,
+MPI_Comm_create must be called by all processes in the group of
+comm, whereas MPI_Comm_create_group must be called by all processes in group,
+which is a subgroup of the group of \fIcomm\fP. In addition, MPI_Comm_create_group
+requires that \fIcomm\fP is an intracommunicator. MPI_Comm_create_group returns a new
+intracommunicator, \fInewcomm\fP, for which the group argument defines the communication
+group. No cached information propagates from \fIcomm\fP to \fInewcomm\fP.
+
+Each process must provide a group argument that is a subgroup of the group associated with \fIcomm\fP;
+this could be MPI_GROUP_EMPTY. If a non-empty group is specified, then all processes in that
+group must call the function, and each of these processes must provide the same arguments,
+including a group that contains the same members with the same ordering. Otherwise
+the call is erroneous. If the calling process is a member of the group given as the \fIgroup\fP
+argument, then newcomm is a communicator with group as its associated group. If the
+calling process is not a member of group, e.g., \fIgroup\fP is MPI_GROUP_EMPTY, then the call
+is a local operation and MPI_COMM_NULL is returned as \fInewcomm\fP.
+
+.sp
+.LP
+
+.SH NOTES
+MPI_Comm_create_group provides a means of making a subset of processes for the purpose of separate MIMD computation, with separate communication space. \fInewcomm\fR, which is created by MPI_Comm_create_group, can be used in subsequent calls to MPI_Comm_create_group (or other communicator constructors) to further subdivide a computation into parallel sub-computations. A more general service is provided by MPI_Comm_split.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+MPI_Comm_create
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_create_keyval.3

@@ -0,0 +1,119 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_create_keyval 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_create_keyval\fP \- Generates a new attribute key.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_create_keyval(MPI_Comm_copy_attr_function
+	*\fIcomm_copy_attr_fn\fP, MPI_Comm_delete_attr_function
+	*\fIcomm_delete_attr_fn\fP, int *\fIcomm_keyval\fP,
+	void *\fIextra_state\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm_copy_attr_fn
+Copy callback function for \fIcomm_keyval\fP (function).
+.TP 1i
+comm_delete_attr_fn
+Delete callback function for \fIcomm_keyval\fP (function).
+.TP 1i
+extra_state
+Extra state for callback functions.
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm_keyval
+Key value for future access (integer).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function replaces MPI_Keyval_create, the use of which is deprecated. The C binding is identical. The Fortran binding differs in that \fIextra_state\fP is an address-sized integer. Also, the copy and delete callback functions have Fortran bindings that are consistent with address-sized attributes.
+.sp
+The argument \fIcomm_copy_attr_fn\fP may be specified as MPI_COMM_NULL_COPY_FN or MPI_COMM_DUP_FN from C, C++, or Fortran. MPI_COMM_NULL_COPY_FN is a function that does nothing more than returning \fIflag\fP = 0 and MPI_SUCCESS. MPI_COMM_DUP_FN is a simple-minded copy function that sets \fIflag\fP = 1, returns the value of \fIattribute_val_in\fP in \fIattribute_val_out\fP, and returns MPI_SUCCESS. These replace the MPI-1 predefined callbacks MPI_NULL_COPY_FN and MPI_DUP_FN, the use of which is deprecated.
+.sp
+The C callback functions are:
+.sp
+.nf
+typedef int MPI_Comm_copy_attr_function(MPI_Comm \fIoldcomm\fP, int \fIcomm_keyval\fP,
+             void *\fIextra_state\fP, void *\fIattribute_val_in\fP,
+             void *\fIattribute_val_out\fP, int *\fIflag\fP);
+.fi
+and
+.nf
+typedef int MPI_Comm_delete_attr_function(MPI_Comm \fIcomm\fP, int \fIcomm_keyval\fP,
+             void *\fIattribute_val\fP, void *\fIextra_state\fP);
+.fi
+.sp
+which are the same as the MPI-1.1 calls but with a new name. The old names are deprecated.
+.sp
+The Fortran callback functions are:
+.sp
+.nf
+SUBROUTINE COMM_COPY_ATTR_FN(\fIOLDCOMM, COMM_KEYVAL, EXTRA_STATE,
+             ATTRIBUTE_VAL_IN, ATTRIBUTE_VAL_OUT, FLAG, IERROR\fP)
+    INTEGER \fIOLDCOMM, COMM_KEYVAL, IERROR\fP
+    INTEGER(KIND=MPI_ADDRESS_KIND) \fIEXTRA_STATE, ATTRIBUTE_VAL_IN,
+        ATTRIBUTE_VAL_OUT\fP
+    LOGICAL \fIFLAG\fP
+.fi
+and
+.nf
+SUBROUTINE COMM_DELETE_ATTR_FN(\fICOMM, COMM_KEYVAL, ATTRIBUTE_VAL, EXTRA_STATE,
+             IERROR\fP)
+    INTEGER \fICOMM, COMM_KEYVAL, IERROR\fP
+    INTEGER(KIND=MPI_ADDRESS_KIND) \fIATTRIBUTE_VAL, EXTRA_STATE\fP
+.fi
+.sp
+The C++ callbacks are:
+.sp
+.nf
+typedef int MPI::Comm::Copy_attr_function(const MPI::Comm& \fIoldcomm\fP,
+             int \fIcomm_keyval\fP, void* \fIextra_state\fP, void* \fIattribute_val_in\fP,
+             void* \fIattribute_val_out\fP, bool& \fIflag\fP);
+.fi
+and
+.nf
+typedef int MPI::Comm::Delete_attr_function(MPI::Comm& \fIcomm\fP,
+             int \fIcomm_keyval\fP, void* \fIattribute_val\fP, void* \fIextra_state\fP);
+.fi
+
+.SH FORTRAN 77 NOTES
+.ft R
+The MPI standard prescribes portable Fortran syntax for
+the \fIEXTRA_STATE\fP argument only for Fortran 90. FORTRAN 77
+users may use the non-portable syntax
+.sp
+.nf
+     INTEGER*MPI_ADDRESS_KIND \fIEXTRA_STATE\fP
+.fi
+.sp
+where MPI_ADDRESS_KIND is a constant defined in mpif.h
+and gives the length of the declared integer in bytes.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+.SH SEE ALSO
+.ft R
+.sp
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_delete_attr.3

@@ -0,0 +1,58 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_delete_attr 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_delete_attr\fP \- Deletes attribute value associated with a key.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_delete_attr(MPI_Comm \fIcomm\fP, int \fIcomm_keyval\fP)
+
+.fi
+.SH INPUT/OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator from which the attribute is deleted (handle).
+
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm_keyval
+Key value (integer).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_delete_attr deletes an attribute from cache by key. This function invokes the attribute delete function delete_fn specified when the \fIcomm_keyval\fP was created. The call will fail if the delete_fn function returns an error code other than MPI_SUCCESS.
+
+Whenever a communicator is replicated using the function MPI_Comm_dup, all callback copy functions for attributes that are currently set are invoked (in arbitrary order). Whenever a communicator is deleted using the function MPI_Comm_free, all callback delete functions for attributes that are currently set are invoked.
+.sp
+This function is the same as MPI_Attr_delete but is needed to match the communicator-specific functions introduced in the MPI-2 standard. The use of MPI_Attr_delete is deprecated.
+
+
+.SH NOTES
+Note that it is not defined by the MPI standard what happens if the
+delete_fn callback invokes other MPI functions.  In Open MPI, it is
+not valid for delete_fn callbacks (or any of their children) to add or
+delete attributes on the same object on which the delete_fn callback
+is being invoked.
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_disconnect.3

@@ -0,0 +1,56 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_disconnect 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_disconnect\fP \- Deallocates communicator object and sets handle to MPI_COMM_NULL.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_disconnect(MPI_Comm *\fIcomm\fP)
+
+.fi
+.SH INPUT/OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_disconnect waits for all pending communication on \fIcomm\fP to complete internally, deallocates the communicator object, and sets the handle to MPI_COMM_NULL. It is a collective operation.
+.sp
+It may not be called with the communicator MPI_COMM_WORLD or MPI_COMM_SELF.
+.sp
+MPI_Comm_disconnect may be called only if all communication is complete and matched, so that buffered data can be delivered to its destination. This requirement is the same as for MPI_Finalize.
+.sp
+MPI_Comm_disconnect has the same action as MPI_Comm_free, except that it waits for pending communication to finish internally and enables the guarantee about the behavior of disconnected processes.
+
+.SH NOTES
+.ft R
+To disconnect two processes you may need to call MPI_Comm_disconnect, MPI_Win_free, and MPI_File_close to remove all communication paths between the two processes. Note that it may be necessary to disconnect several communicators (or to free several windows or files) before two processes are completely independent.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Comm_connect
+.br
+MPI_Comm_accept
+.br

macx64/mpi/openmpi/share/man/man3/MPI_Comm_dup.3

@@ -0,0 +1,62 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_dup 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_dup \fP \-  Duplicates an existing communicator with all its cached information.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_dup(MPI_Comm \fIcomm\fP, MPI_Comm\fI *newcomm\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+Copy of comm (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_dup duplicates the existing communicator comm with associated key
+values. For each key value, the respective copy callback function determines the attribute value associated with this key in the new communicator; one particular action that a copy callback may take is to delete the attribute from the new communicator. Returns in newcomm a new communicator with the same group, any copied cached information, but a new context (see Section 5.7.1 of the MPI-1 Standard, "Functionality").
+
+.SH NOTES
+This operation is used to provide a parallel
+library call with a duplicate communication space that has the same properties as the original communicator. This includes any attributes (see below) and topologies (see Chapter 6, "Process Topologies," in the MPI-1 Standard). This call is valid even if there are pending point-to-point communications involving the communicator comm. A typical call might involve an MPI_Comm_dup at the beginning of the parallel call, and an MPI_Comm_free of that duplicated communicator at the end of the call. Other models of communicator management are also possible.
+.sp
+This call applies to both intra- and intercommunicators.
+
+Note that it is not defined by the MPI standard what happens if the
+attribute copy callback invokes other MPI functions.  In Open MPI, it
+is not valid for attribute copy callbacks (or any of their children)
+to add or delete attributes on the same object on which the attribute
+copy callback is being invoked.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Comm_dup_with_info
+MPI_Comm_idup

macx64/mpi/openmpi/share/man/man3/MPI_Comm_dup_with_info.3

@@ -0,0 +1,71 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_dup_with_info 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_dup_with_info \fP \-  Duplicates an existing communicator using provided info.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_dup_with_info(MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP, MPI_Comm\fI *newcomm\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+.ft R
+.TP 1i
+info
+Info argument (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+Copy of comm (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_dup_with_info acts exactly like MPI_Comm_dup except that the
+info hints associated with the communicator \fIcomm\fP are not duplicated in \fInewcomm\fP. The
+hints provided by the argument \fIinfo\fP are associated with the output communicator \fInewcomm\fP
+instead.
+.sp
+See
+.BR MPI_Comm_set_info (3)
+for the list of recognized info keys.
+
+.SH NOTES
+This operation is used to provide a parallel
+library call with a duplicate communication space that has the same properties as the original communicator. This includes any attributes (see below) and topologies (see Chapter 6, "Process Topologies," in the MPI-1 Standard). This call is valid even if there are pending point-to-point communications involving the communicator comm. A typical call might involve an MPI_Comm_dup_with_info at the beginning of the parallel call, and an MPI_Comm_free of that duplicated communicator at the end of the call. Other models of communicator management are also possible.
+.sp
+This call applies to both intra- and intercommunicators.
+
+Note that it is not defined by the MPI standard what happens if the
+attribute copy callback invokes other MPI functions.  In Open MPI, it
+is not valid for attribute copy callbacks (or any of their children)
+to add or delete attributes on the same object on which the attribute
+copy callback is being invoked.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+MPI_Comm_dup
+MPI_Comm_idup
+MPI_Comm_set_info

macx64/mpi/openmpi/share/man/man3/MPI_Comm_f2c.3

@@ -0,0 +1,48 @@
+.\" -*- nroff -*-
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_f2c 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_f2c, MPI_Comm_c2f, MPI_File_f2c, MPI_File_c2f, MPI_Info_f2c, MPI_Info_c2f, MPI_Message_f2c, MPI_Message_c2f, MPI_Op_f2c, MPI_Op_c2f, MPI_Request_f2c, MPI_Request_c2f, MPI_Type_f2c, MPI_Type_c2f, MPI_Win_f2c, MPI_Win_c2f \fP \- Translates a C handle into a Fortran handle, or vice versa.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+MPI_Comm MPI_Comm_f2c(MPI_Fint \fIcomm\fP)
+MPI_Fint MPI_Comm_c2f(MPI_Comm \fIcomm\fP)
+
+MPI_File MPI_File_f2c(MPI_Fint \fIfile\fP)
+MPI_Fint MPI_File_c2f(MPI_File \fIfile\fP)
+
+MPI_Group MPI_Group_f2c(MPI Fint \fIgroup\fP)
+MPI_Fint MPI_Group_c2f(MPI Group \fIgroup\fP)
+
+MPI_Info MPI_Info_f2c(MPI_Fint \fIinfo\fP)
+MPI_Fint MPI_Info_c2f(MPI_Info \fIinfo\fP)
+
+MPI_Message MPI_Message_f2c(MPI_Fint \fImessage\fP)
+MPI_Fint MPI_Message_c2f(MPI_Message \fImessage\fP)
+
+MPI_Op MPI_Op_f2c(MPI_Fint \fIop\fP)
+MPI_Fint MPI_Op_c2f(MPI_Op \fIop\fP)
+
+MPI_Request MPI_Request_f2c(MPI_Fint \fIrequest\fP)
+MPI_Fint MPI_Request_c2f(MPI_Request \fIrequest\fP)
+
+MPI_Datatype MPI_Type_f2c(MPI_Fint \fIdatatype\fP)
+MPI_Fint MPI_Type_c2f(MPI_Datatype \fIdatatype\fP)
+
+MPI_Win MPI_Win_f2c(MPI_Fint \fIwin\fP)
+MPI_Fint MPI_Win_c2f(MPI_Win \fIwin\fP)
+
+.fi
+.SH DESCRIPTION
+.ft R
+Handles are passed between Fortran and C or C++ by using an explicit C wrapper to convert Fortran handles to C handles. There is no direct access to C or C++ handles in Fortran. Handles are passed between C and C++ using overloaded C++ operators called from C++ code. There is no direct access to C++ objects from C. The type definition \fIMPI_Fint\fP is provided in C/C++ for an integer of the size that matches a Fortran \fIINTEGER\fP; usually, \fIMPI_Fint\fP will be equivalent to \fIint\fP. The handle translation functions are provided in C to convert from a Fortran handle (which is an integer) to a C handle, and vice versa.
+.PP
+For example, if \fIcomm\fP is a valid Fortran handle to a communicator, then MPI_Comm_f2c returns a valid C handle to that same communicator; if \fIcomm\fP = MPI_COMM_NULL (Fortran value), then MPI_Comm_f2c returns a null C handle; if \fIcomm\fP is an invalid Fortran handle, then MPI_Comm_f2c returns an invalid C handle.
+.SH NOTE
+This function does not return an error value. Consequently, the result of calling it before MPI_Init or after MPI_Finalize is undefined.

macx64/mpi/openmpi/share/man/man3/MPI_Comm_free.3

@@ -0,0 +1,52 @@
+.\" -*- nroff -*-
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_free 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_free \fP \- Mark a communicator object for deallocation.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_free(MPI_Comm *\fIcomm\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator to be destroyed (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This operation marks the communicator object for deallocation. The handle is set to MPI_COMM_NULL. Any pending operations that use this communicator will complete normally; the object is actually deallocated only if there are no other active references to it. This call applies to intracommunicators and intercommunicators. Upon actual deallocation, the delete callback functions for all cached attributes (see Section 5.7 in the MPI-1 Standard, "Caching") are called in arbitrary order.
+
+
+.SH NOTES
+Note that it is not defined by the MPI standard what happens if the
+delete_fn callback invokes other MPI functions.  In Open MPI, it is
+not valid for delete_fn callbacks (or any of their children) to add or
+delete attributes on the same object on which the delete_fn callback
+is being invoked.
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.nf
+MPI_Comm_delete_attr

macx64/mpi/openmpi/share/man/man3/MPI_Comm_free_keyval.3

@@ -0,0 +1,46 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_free_keyval 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_free_keyval\fP \- Frees attribute key for communicator cache attribute.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_free_keyval(int *\fIcomm_keyval\fP)
+
+.fi
+.SH INPUT/OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm_keyval
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_free_keyval frees an extant attribute key. This function sets the value of \fIkeyval\fP to  MPI_KEYVAL_INVALID. Note that it is not erroneous to free an attribute key that is in use, because the actual free does not transpire until after all references (in other communicators on the process) to the key have been freed. These references need to be explicitly freed by the program, either via calls to MPI_Comm_delete_attr that free one attribute instance, or by calls to MPI_Comm_free that free all attribute instances associated with the freed communicator.
+.sp
+This call is identical to the call MPI_Keyval_free but is needed to match the  communicator-specific creation function introduced in the MPI-2 standard. The use of MPI_Keyval_free is deprecated.
+
+
+.SH NOTES
+.ft R
+Key values are global (they can be used with any and all communicators).
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_get_attr.3

@@ -0,0 +1,64 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_get_attr 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_get_attr\fP \- Retrieves attribute value by key.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_get_attr(MPI_Comm \fIcomm\fP, int \fIcomm_keyval\fP,
+	void *\fIattribute_val\fP, int *\fIflag\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator to which the attribute is attached (handle).
+.TP 1i
+comm_keyval
+Key value (integer).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+attribute_val
+Attribute value, unless f\fIlag\fP = false.
+.TP 1i
+flag
+False if no attribute is associated with the key (logical).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_get_attr retrieves an attribute value by key. The call is erroneous if there is no key with value \fIkeyval\fP. On the other hand, the call is correct if the key value exists, but no attribute is attached on \fIcomm\fP for that key; in that case, the call returns \fIflag\fP = false. In particular, MPI_KEYVAL_INVALID is an erroneous key value.
+.sp
+This function replaces MPI_Attr_get, the use of which is deprecated. The C binding is identical. The Fortran binding differs in that \fIattribute_val\fP is an address-sized integer.
+
+.SH FORTRAN 77 NOTES
+.ft R
+The MPI standard prescribes portable Fortran syntax for
+the \fIATTRIBUTE_VAL\fP argument only for Fortran 90. Sun FORTRAN 77
+users may use the non-portable syntax
+.sp
+.nf
+     INTEGER*MPI_ADDRESS_KIND \fIATTRIBUTE_VAL\fP
+.fi
+.sp
+where MPI_ADDRESS_KIND is a constant defined in mpif.h
+and gives the length of the declared integer in bytes.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_get_errhandler.3

@@ -0,0 +1,47 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_get_errhandler 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_get_errhandler \fP \- Retrieves error handler associated with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_get_errhandler(MPI_Comm \fIcomm\fP,
+	MPI_Errhandler *\fIerrhandler\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+errhandler
+New error handler for communicator (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_get_errhandler retrieves the error handler currently associated with a communicator. This call is identical to MPI_Errhandler_get, the use of which is deprecated.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_get_info.3

@@ -0,0 +1,59 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_get_info 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_get_info\fP \- Retrieves active communicator info hints
+.
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_get_info(MPI_Comm \fIcomm\fP, MPI_Info \fI*info_used\fP)
+.
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator from which to receive active info hints
+.
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+info_used
+New info object returned with all active hints on this communicator.
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+.
+.SH DESCRIPTION
+.ft R
+MPI_Comm_get_info returns a new info object containing the hints of
+the communicator associated with
+.IR comm .
+The current setting of all hints actually used by the system related
+to this communicator is returned in
+.IR info_used .
+If no such hints exist, a handle to a newly created info object is
+returned that contains no key/value pair. The user is responsible for
+freeing info_used via MPI_Info_free.
+.
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value
+of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler
+MPI_ERRORS_RETURN may be used to cause error values to be
+returned. Note that MPI does not guarantee that an MPI program can
+continue past an error.
+.
+.SH SEE ALSO
+MPI_Comm_get_info,
+MPI_Info_free

macx64/mpi/openmpi/share/man/man3/MPI_Comm_get_name.3

@@ -0,0 +1,54 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_get_name 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_get_name\fP \- Returns the name that was most recently associated with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_get_name(MPI_Comm \fIcomm\fP, char *\fIcomm_name\fP, int *\fIresultlen\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator the name of which is to be returned (handle).
+.TP 1i
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm_name
+Name previously stored on the communicator, or an empty string if no such name exists (string).
+.TP 1i
+resultlen
+Length of returned name (integer).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_get_name returns the last name that was previously associated with the given communicator. The name may be set and retrieved from any language. The same name will be returned independent of the language used. \fIcomm_name\fP should be allocated so that it can hold a resulting string of length MPI_MAX_OBJECT_NAME characters. MPI_Comm_get_name returns a copy of the set name in \fIcomm_name\fP.
+.sp
+If the user has not associated a name with a communicator, or an error occurs, MPI_Comm_get_name will return an empty string (all spaces in Fortran, "" in C and C++). The three predefined communicators will have predefined names associated with them. Thus, the names of MPI_COMM_WORLD, MPI_COMM_SELF, and MPI_COMM_PARENT will have the default of MPI_COMM_WORLD, MPI_COMM_SELF, and MPI_COMM_PARENT. The fact that the system may have chosen to give a default name to a communicator does not prevent the user from setting a name on the same communicator; doing this removes the old name and assigns the new one.
+
+.SH NOTES
+.ft R
+It is safe simply to print the string returned by MPI_Comm_get_name, as it is always a valid string even if there was no name.
+.sp
+Note that associating a name with a communicator has no effect on the semantics of an MPI program, and will (necessarily) increase the store requirement of the program, since the names must be saved. Therefore, there is no requirement that users use these functions to associate names with communicators. However debugging and profiling MPI applications may be made easier if names are associated with communicators, since the debugger or profiler should then be able to present information in a less cryptic manner.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_get_parent.3

@@ -0,0 +1,51 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_get_parent 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_get_parent\fP \- Returns the parent intercommunicator of current spawned process.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_get_parent(MPI_Comm *\fIparent\fP)
+
+.fi
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+parent
+The parent communicator (handle).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+If a process was started with MPI_Comm_spawn or MPI_Comm_spawn_multiple, MPI_Comm_get_parent returns the "parent" intercommunicator of the current process. This parent intercommunicator is created implicitly inside of MPI_Init and is the same intercommunicator returned by the spawn call made in the parents.
+.sp
+If the process was not spawned, MPI_Comm_get_parent returns MPI_COMM_NULL.
+.sp
+After the parent communicator is freed or disconnected, MPI_Comm_get_parent returns MPI_COMM_NULL.
+
+.SH NOTES
+.ft R
+MPI_Comm_get_parent returns a handle to a single intercommunicator. Calling MPI_Comm_get_parent a second time returns a handle to the same intercommunicator. Freeing the handle with MPI_Comm_disconnect or MPI_Comm_free will cause other references to the intercommunicator to become invalid (dangling). Note that calling MPI_Comm_free on the parent communicator is not useful.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Comm_spawn
+MPI_Comm_spawn_multiple
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_group.3

@@ -0,0 +1,43 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_group 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_group \fP \- Returns the group associated with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_group(MPI_Comm \fIcomm\fP, MPI_Group *\fIgroup\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator.
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+group
+Group in communicator (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+If the communicator is an intercommunicator (enables communication between two groups of processes), this function returns the local group.  To return the remote group, use the MPI_Comm_remote_group function.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_idup.3

@@ -0,0 +1,67 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2014 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_idup 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_idup \fP \-  Start the nonblocking duplication of an existing communicator with all its cached information.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_idup(MPI_Comm \fIcomm\fP, MPI_Comm\fI *newcomm\fP, MPI_Request\fI *request\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+Copy of comm (handle).
+.ft R
+.TP 1i
+request
+Communication request (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_idup starts the nonblocking duplication of an existing communicator comm with associated key
+values. For each key value, the respective copy callback function determines the attribute value associated with this key in the new communicator; one particular action that a copy callback may take is to delete the attribute from the new communicator. Returns in newcomm a new communicator with the same group, any copied cached information, but a new context (see Section 5.7.1 of the MPI-1 Standard, "Functionality"). The communicator returned in \fInewcomm\fP will not be available until the request is complete.
+.sp
+The completion of a communicator duplication request can be determined by calling any of MPI_Wait, MPI_Waitany, MPI_Test, or MPI_Testany with the request returned by this function.
+
+.SH NOTES
+This operation is used to provide a parallel
+library call with a duplicate communication space that has the same properties as the original communicator. This includes any attributes (see below) and topologies (see Chapter 6, "Process Topologies," in the MPI-1 Standard). This call is valid even if there are pending point-to-point communications involving the communicator comm. A typical call might involve an MPI_Comm_idup at the beginning of the parallel call, and an MPI_Comm_free of that duplicated communicator at the end of the call. Other models of communicator management are also possible.
+.sp
+This call applies to both intra- and intercommunicators.
+
+Note that it is not defined by the MPI standard what happens if the
+attribute copy callback invokes other MPI functions.  In Open MPI, it
+is not valid for attribute copy callbacks (or any of their children)
+to add or delete attributes on the same object on which the attribute
+copy callback is being invoked.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+
+.SH SEE ALSO
+MPI_Comm_dup
+MPI_Comm_dup_with_info

macx64/mpi/openmpi/share/man/man3/MPI_Comm_join.3

@@ -0,0 +1,89 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_join 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+
+.SH NAME
+\fBMPI_Comm_join\fP \- Establishes communication between MPI jobs
+
+.SH SYNTAX
+.ft R
+
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_join(int \fIfd\fP, MPI_Comm *\fIintercomm\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+fd
+socket file descriptor (socket).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+intercomm
+Intercommunicator between processes (handle).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_join creates an intercommunicator from the union of two MPI
+processes that are connected by a socket. \fIfd\fP is a file
+descriptor representing a socket of type SOCK_STREAM (a two-way
+reliable byte-stream connection). Nonblocking I/O and asynchronous
+notification via SIGIO must not be enabled for the socket. The socket
+must be in a connected state, and must be quiescent when MPI_Comm_join
+is called.
+.sp
+MPI_Comm_join must be called by the process at each end of the
+socket. It does not return until both processes have called
+MPI_Comm_join.
+
+.SH NOTES
+.ft R
+There are no MPI library calls for opening and manipulating a socket.
+The socket \fIfd\fP can be opened using standard socket API calls.
+MPI uses the socket to bootstrap creation of the intercommunicator,
+and for nothing else. Upon return, the file descriptor will be open
+and quiescent.
+.sp
+In a multithreaded process, the application must ensure that other
+threads do not access the socket while one is in the midst of
+calling MPI_Comm_join.
+.sp
+The returned communicator will contain the two processes connected by
+the socket, and may be used to establish MPI communication with
+additional processes, through the usual MPI communicator-creation
+mechanisms.
+
+.SH ERRORS
+.ft R
+Almost all MPI routines return an error value; C routines as
+the value of the function and Fortran routines in the last argument. C++
+functions do not return errors. If the default error handler is set to
+MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism
+will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN
+may be used to cause error values to be returned. Note that MPI does not
+guarantee that an MPI program can continue past an error.
+.sp
+See the MPI man page for a full list of MPI error codes.
+
+.SH SEE ALSO
+.ft R
+.nf
+socket(3SOCKET)
+MPI_Comm_create
+MPI_Comm_group
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_rank.3

@@ -0,0 +1,55 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_rank 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_rank\fP \- Determines the rank of the calling process in the communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_rank(MPI_Comm \fIcomm\fP, int\fI *rank\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+rank
+Rank of the calling process in group of comm (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function gives the rank of the process in the
+particular communicator's group. It is equivalent to accessing the
+communicator's group with MPI_Comm_group, computing the rank using MPI_Group_rank, and then freeing the temporary group via MPI_Group_free.
+.sp
+Many programs will be written with the master-slave model, where one process (such as the rank-zero process) will play a supervisory role, and the other processes will serve as compute nodes. In this framework, MPI_Comm_size and MPI_Comm_rank are useful for determining the roles of the various processes of a communicator.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Comm_group
+MPI_Comm_size
+MPI_Comm_compare
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_remote_group.3

@@ -0,0 +1,54 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_remote_group 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_remote_group \fP \- Accesses the remote group associated with an intercommunicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_remote_group(MPI_Comm \fIcomm\fP, MPI_Group\fI *group\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator.
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+group
+Remote group of communicator.
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_remote_group accesses the remote group associated with an intercommunicator.
+.sp
+The intercommunicator accessors (MPI_Comm_test_inter, MPI_Comm_remote_size,
+MPI_Comm_remote_group) are all local operations.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.sp
+.nf
+MPI_Comm_test_inter
+MPI_Comm_remote_size
+MPI_Intercomm_create
+MPI_Intercomm_merge
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_remote_size.3

@@ -0,0 +1,54 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_remote_size 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_remote_size \fP \- Determines the size of the remote group associated with an intercommunicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_remote_size(MPI_Comm \fIcomm\fP, int\fI *size\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+size
+Number of processes in the remote group of comm (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_remote_size determines the size of the remote group associated with an intercommunicator.
+.sp
+The  intercommunicator accessors (MPI_Comm_test_inter, MPI_Comm_remote_size, MPI_Comm_remote_group) are all local operations.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Comm_test_inter
+MPI_Comm_remote_group
+MPI_Intercomm_create
+MPI_Intercomm_merge
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_set_attr.3

@@ -0,0 +1,72 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_set_attr 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_set_attr\fP \- Stores attribute value associated with a key.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_set_attr(MPI_Comm \fIcomm\fP, int \fIcomm_keyval\fP, void *\fIattribute_val\fP)
+
+.fi
+.SH INPUT/OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator from which attribute will be attached (handle).
+
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm_keyval
+Key value (integer).
+.TP 1i
+attribute_val
+Attribute value.
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_set_attr stores the stipulated attribute value \fIattribute_val\fP for subsequent retrieval by MPI_Comm_get_attr. If the value is already present, then the outcome is as if MPI_Comm_delete_attr was first called to delete the previous value (and the callback function delete_fn was executed), and a new value was next stored. The call is erroneous if there is no key with value \fIcomm_keyval\fP; in particular MPI_KEYVAL_INVALID is an erroneous key value. The call will fail if the delete_fn function returned an error code other than MPI_SUCCESS.
+.sp
+This function replaces MPI_Attr_put, the use of which is deprecated. The C binding is identical. The Fortran binding differs in that \fIattribute_val\fP is an address-sized integer.
+
+.SH FORTRAN 77 NOTES
+.ft R
+The MPI standard prescribes portable Fortran syntax for
+the \fIATTRIBUTE_VAL\fP argument only for Fortran 90. Sun FORTRAN 77
+users may use the non-portable syntax
+.sp
+.nf
+     INTEGER*MPI_ADDRESS_KIND \fIATTRIBUTE_VAL\fP
+.fi
+.sp
+where MPI_ADDRESS_KIND is a constant defined in mpif.h
+and gives the length of the declared integer in bytes.
+
+.SH NOTES
+.ft R
+Values of the permanent attributes MPI_TAG_UB, MPI_HOST,
+MPI_IO, and MPI_WTIME_IS_GLOBAL may not be changed.
+.sp
+The type of the attribute value depends on whether C or Fortran is being used. In C, an attribute value is a pointer (void *); in Fortran, it is a single, address-size integer system for which a pointer does not fit in an integer.
+.sp
+If an attribute is already present, the delete function (specified when the corresponding keyval was created) will be called.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_set_errhandler.3

@@ -0,0 +1,44 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_set_errhandler 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_set_errhandler \fP \- Attaches a new error handler to a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_set_errhandler(MPI_Comm \fIcomm\fP,
+	MPI_Errhandler \fIerrhandler\fP)
+
+.fi
+.SH INPUT/OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+errhandler
+New error handler for communicator (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_set_errhandler attaches a new error handler to a communicator. The error handler must be either a predefined error handler or an error handler created by a call to MPI_Comm_create_errhandler. This call is identical to MPI_Errhandler_set, the use of which is deprecated.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_set_info.3

@@ -0,0 +1,86 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_set_info 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_set_info\fP \- Set communicator info hints
+.
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_set_info(MPI_Comm \fIcomm\fP, MPI_Info \fIinfo\fP)
+.
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator on which to set info hints
+.TP 1i
+info
+Info object containing hints to be set on
+.I comm
+.
+.SH OUTPUT PARAMETERS
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+.
+.SH DESCRIPTION
+.ft R
+MPI_COMM_SET_INFO sets new values for the hints of the communicator
+associated with
+.IR comm .
+MPI_COMM_SET_INFO is a collective routine. The info object may be
+different on each process, but any info entries that an implementation
+requires to be the same on all processes must appear with the same
+value in each process's
+.I info
+object.
+.sp
+The following info key assertions may be accepted by Open MPI:
+.sp
+\fImpi_assert_no_any_tag\fP (boolean): If set to true, then the
+implementation may assume that the process will not use the
+MPI_ANY_TAG wildcard on the given
+communicator.
+.sp
+\fImpi_assert_no_any_source\fP (boolean): If set to true, then
+the implementation may assume that the process will not use the
+MPI_ANY_SOURCE wildcard on the given communicator.
+.sp
+\fImpi_assert_exact_length\fP (boolean): If set to true, then the
+implementation may assume that the lengths of messages received by the
+process are equal to the lengths of the corresponding receive buffers,
+for point-to-point communication operations on the given communicator.
+.sp
+\fImpi_assert_allow_overtaking\fP (boolean): If set to true, then the
+implementation may assume that point-to-point communications on the
+given communicator do not rely on the non-overtaking rule specified in
+MPI-3.1 Section 3.5. In other words, the application asserts that send
+operations are not required to be matched at the receiver in the order
+in which the send operations were performed by the sender, and receive
+operations are not required to be matched in the order in which they
+were performed by the receiver.
+.
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value
+of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler
+MPI_ERRORS_RETURN may be used to cause error values to be
+returned. Note that MPI does not guarantee that an MPI program can
+continue past an error.
+.
+.SH SEE ALSO
+MPI_Comm_get_info,
+MPI_Info_create,
+MPI_Info_set,
+MPI_Info_free

macx64/mpi/openmpi/share/man/man3/MPI_Comm_set_name.3

@@ -0,0 +1,64 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines
+.\" $COPYRIGHT$
+.TH MPI_Comm_set_name 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_set_name\fP \- Associates a name with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_set_name(MPI_Comm \fIcomm\fP, const char *\fIcomm_name\fP)
+
+.fi
+.SH INPUT/OUTPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator whose identifier is to be set (handle).
+
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm_name
+Character string to be used as the identifier for the communicator (string).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_set_name allows a user to associate a name string with a communicator. The character string that is passed to MPI_Comm_set_name is saved inside the MPI library (so it can be freed by the caller immediately after the call, or allocated on the stack). Leading spaces in \fIname\fP are significant, but trailing ones are not.
+.sp
+MPI_Comm_set_name is a local (noncollective) operation, which affects only the name of the communicator as seen in the process that made the MPI_Comm_set_name call. There is no requirement that the same (or any) name be assigned to a communicator in every process where it exists.
+.sp
+The length of the name that can be stored is limited to the value of MPI_MAX_OBJECT_NAME in Fortran and MPI_MAX_OBJECT_NAME-1 in C and C++ (to allow for the null terminator). Attempts to set names longer than this will result in truncation of the name. MPI_MAX_OBJECT_NAME must have a value of at least 64.
+
+
+.SH NOTES
+.ft R
+Since MPI_Comm_set_name is provided to help debug code, it is sensible to give the same name to a communicator in all of the processes where it exists, to avoid confusion.
+.sp
+Regarding name length, under circumstances of store exhaustion, an attempt to set a name of any length could fail; therefore, the value of MPI_MAX_OBJECT_NAME should be viewed only as a strict upper bound on the name length, not a guarantee that setting names of less than this length will always succeed.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+MPI_Comm_get_name
+.sp
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_size.3

@@ -0,0 +1,65 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_size 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_size \fP \- Returns the size of the group associated with a communicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_size(MPI_Comm \fIcomm\fP, int *\fIsize\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+size
+Number of processes in the group of comm (integer).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function indicates the number of processes involved in a
+communicator. For MPI_COMM_WORLD, it indicates the total number of
+processes available. This function is equivalent to accessing the
+communicator's group with MPI_Comm_group, computing the size using
+MPI_Group_size, and then freeing the temporary group via
+MPI_Group_free. If the communicator is an inter-communicator (enables
+communication between two groups), this function returns the size of
+the local group.  To return the size of the remote group, use the
+MPI_Comm_remote_size function.
+.sp
+This call is often used with MPI_Comm_rank to determine the amount of concurrency available for a specific library or program. MPI_Comm_rank indicates the rank of the process that calls it in the range from  0 . . . size-1, where size is the return value of MPI_Comm_size.
+
+.SH NOTE
+.ft R
+MPI_COMM_NULL is not considered a valid argument to this function.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Comm_group
+MPI_Comm_rank
+MPI_Comm_compare
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_spawn.3

@@ -0,0 +1,207 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_spawn 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_spawn\fP \- Spawns a number of identical binaries.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_spawn(const char *\fIcommand\fP, char *\fIargv\fP[], int \fImaxprocs\fP,
+	MPI_Info \fIinfo\fP, int \fIroot\fP, MPI_Comm \fIcomm\fP,
+	MPI_Comm *\fIintercomm\fP, int \fIarray_of_errcodes\fP[])
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+command
+Name of program to be spawned (string, significant only at \fIroot\fP).
+.TP 1i
+argv
+Arguments to \fIcommand\fP (array of strings, significant only at \fIroot\fP).
+.TP 1i
+maxprocs
+Maximum number of processes to start (integer, significant only at \fIroot\fP).
+.TP 1i
+info
+A set of key-value pairs telling the runtime system where and how to start the processes (handle, significant only at \fIroot\fP).
+.TP 1i
+root
+Rank of process in which previous arguments are examined (integer).
+.TP 1i
+comm
+Intracommunicator containing group of spawning processes (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+intercomm
+Intercommunicator between original group and the newly spawned group (handle).
+.TP 1i
+array_of_errcodes
+One code per process (array of integers).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_spawn tries to start \fImaxprocs\fP identical copies of the MPI program specified by \fIcommand\fP, establishing communication with them and returning an intercommunicator. The spawned processes are referred to as children. The children have their own MPI_COMM_WORLD, which is separate from that of the parents. MPI_Comm_spawn is collective over \fIcomm\fP, and also may not return until MPI_Init has been called in the children. Similarly, MPI_Init in the children may not return until all parents have called MPI_Comm_spawn. In this sense, MPI_Comm_spawn in the parents and MPI_Init in the children form a collective operation over the union of parent and child processes. The intercommunicator returned by MPI_Comm_spawn contains the parent processes in the local group and the child processes in the remote group. The ordering of processes in the local and remote groups is the same as the as the ordering of the group of the \fIcomm\fP in the parents and of MPI_COMM_WORLD of the children, respectively. This intercommunicator can be obtained in the children through the function MPI_Comm_get_parent.
+.sp
+The MPI standard allows an implementation to use the MPI_UNIVERSE_SIZE attribute of MPI_COMM_WORLD to specify the number of processes that will be active in a program.  Although this implementation of the MPI standard defines MPI_UNIVERSE_SIZE, it does not allow the user to set its value.  If you try to set the value of MPI_UNIVERSE_SIZE, you will get an error message.
+.sp
+The \fIcommand\fP Argument
+.sp
+The \fIcommand\fP argument is a string containing the name of a program to be spawned. The string is null-terminated in C. In Fortran, leading and trailing spaces are stripped. MPI looks for the file first in the working directory of the spawning process.
+.sp
+The \fIargv\fP Argument
+.sp
+\fIargv\fP is an array of strings containing arguments that are passed
+to the program. The first element of \fIargv\fP is the first argument
+passed to \fIcommand\fP, not, as is conventional in some contexts, the
+command itself. The argument list is terminated by NULL in C and C++
+and an empty string in Fortran (note that it is the MPI application's
+responsibility to ensure that the last entry of the
+.I argv
+array is an empty string; the compiler will not automatically insert
+it). In Fortran, leading and trailing spaces are always stripped, so
+that a string consisting of all spaces is considered an empty
+string. The constant MPI_ARGV_NULL may be used in C, C++ and Fortran
+to indicate an empty argument list. In C and C++, this constant is the
+same as NULL.
+.sp
+In C, the MPI_Comm_spawn argument \fIargv\fP differs from the \fIargv\fP argument of \fImain\fP in two respects. First, it is shifted by one element. Specifically, \fIargv\fP[0] of \fImain\fP  contains the name of the program (given by \fIcommand\fP). \fIargv\fP[1] of \fImain\fP corresponds to \fIargv\fP[0] in MPI_Comm_spawn, \fIargv\fP[2] of \fImain\fP to \fIargv\fP[1] of MPI_Comm_spawn, and so on. Second, \fIargv\fP of MPI_Comm_spawn must be null-terminated, so that its length can be determined. Passing an \fIargv\fP of MPI_ARGV_NULL to MPI_Comm_spawn results in \fImain\fP receiving \fIargc\fP of 1 and an \fIargv\fP whose element 0 is the name of the program.
+.sp
+The \fImaxprocs\fP Argument
+.sp
+Open MPI tries to spawn \fImaxprocs\fP processes. If it is unable to spawn \fImaxprocs\fP processes, it raises an error of class MPI_ERR_SPAWN. If MPI is able to spawn the specified number of processes, MPI_Comm_spawn returns successfully and the number of spawned processes, \fIm\fP, is given by the size of the remote group of \fIintercomm\fP.
+.sp
+A spawn call with the default behavior is called hard. A spawn call for which fewer than \fImaxprocs\fP processes may be returned is called soft.
+.sp
+The \fIinfo\fP Argument
+.sp
+The \fIinfo\fP argument is an opaque handle of type MPI_Info in C, MPI::Info in C++ and INTEGER in Fortran. It is a container for a number of user-specified (\fIkey,value\fP) pairs. \fIkey\fP and \fIvalue\fP are strings (null-terminated char* in C, character*(*) in Fortran). Routines to create and manipulate the \fIinfo\fP argument are described in Section 4.10 of the MPI-2 standard.
+.sp
+For the SPAWN calls, \fIinfo\fP provides additional, implementation-dependent instructions to MPI and the runtime system on how to start processes. An application may pass MPI_INFO_NULL in C or Fortran. Portable programs not requiring detailed control over process locations should use MPI_INFO_NULL.
+.sp
+The following keys for \fIinfo\fP are recognized in Open MPI. (The reserved values mentioned in Section 5.3.4 of the MPI-2 standard are not implemented.)
+.sp
+.nf
+Key                    Type     Description
+---                    ----     -----------
+
+host                   char *   Host on which the process should be
+                                spawned.  See the \fIorte_host\fP man
+                                page for an explanation of how this
+                                will be used.
+hostfile               char *   Hostfile containing the hosts on which
+                                the processes are to be spawned. See
+                                the \fIorte_hostfile\fP man page for
+                                an explanation of how this will be
+                                used.
+add-host               char *   Add the specified host to the list of
+                                hosts known to this job and use it for
+                                the associated process. This will be
+                                used similarly to the -host option.
+add-hostfile           char *   Hostfile containing hosts to be added
+                                to the list of hosts known to this job
+                                and use it for the associated
+                                process. This will be used similarly
+                                to the -hostfile option.
+wdir                   char *   Directory where the executable is
+                                located. If files are to be
+                                pre-positioned, then this location is
+                                the desired working directory at time
+                                of execution - if not specified, then
+                                it will automatically be set to
+                                \fIompi_preload_files_dest_dir\fP.
+ompi_prefix            char *   Same as the --prefix command line
+                                argument to mpirun.
+ompi_preload_binary    bool     If set to true, pre-position the
+                                specified executable onto the remote
+                                host. A destination directory must
+                                also be provided.
+ompi_preload_files     char *   A comma-separated list of files that
+                                are to be pre-positioned in addition
+                                to the executable.  Note that this
+                                option does not depend upon
+                                \fIompi_preload_binary\fP - files can
+                                be moved to the target even if an
+                                executable is not moved.
+ompi_stdin_target      char *   Comma-delimited list of ranks to
+                                receive stdin when forwarded.
+ompi_non_mpi           bool     If set to true, launching a non-MPI
+                                application; the returned communicator
+                                will be MPI_COMM_NULL. Failure to set
+                                this flag when launching a non-MPI
+                                application will cause both the child
+                                and parent jobs to "hang".
+ompi_param             char *   Pass an OMPI MCA parameter to the
+                                child job.  If that parameter already
+                                exists in the environment, the value
+                                will be overwritten by the provided
+                                value.
+mapper                 char *   Mapper to be used for this job
+map_by                 char *   Mapping directive indicating how
+                                processes are to be mapped (slot,
+                                node, socket, etc.).
+rank_by                char *   Ranking directive indicating how
+                                processes are to be ranked (slot,
+                                node, socket, etc.).
+bind_to                char *   Binding directive indicating how
+                                processes are to be bound (core, slot,
+                                node, socket, etc.).
+path                   char *   List of directories to search for
+                                the executable
+npernode               char *   Number of processes to spawn on
+                                each node of the allocation
+pernode                bool     Equivalent to npernode of 1
+ppr                    char *   Spawn specified number of processes
+                                on each of the identified object type
+env                    char *   Newline-delimited list of envars to
+                                be passed to the spawned procs
+.fi
+
+\fIbool\fP info keys are actually strings but are evaluated as
+follows: if the string value is a number, it is converted to an
+integer and cast to a boolean (meaning that zero integers are false
+and non-zero values are true).  If the string value is
+(case-insensitive) "yes" or "true", the boolean is true.  If the
+string value is (case-insensitive) "no" or "false", the boolean is
+false.  All other string values are unrecognized, and therefore false.
+
+.sp
+The \fIroot\fP Argument
+.sp
+All arguments before the \fIroot\fP argument are examined only on the process whose rank in \fIcomm\fP is equal to \fIroot\fP. The value of these arguments on other processes is ignored.
+.sp
+The \fIarray_of_errcodes\fP Argument
+.sp
+The \fIarray_of_errcodes\fP is an array of length \fImaxprocs\fP in which MPI reports the status of the processes that MPI was requested to start. If all \fImaxprocs\fP processes were spawned, \fIarray_of_errcodes\fP is filled in with the value MPI_SUCCESS. If anyof the processes are \fInot\fP spawned, \fIarray_of_errcodes\fP is filled in with the value MPI_ERR_SPAWN. In C or Fortran, an application may pass MPI_ERRCODES_IGNORE if it is not interested in the error codes. In C++ this constant does not exist, and the \fIarray_of_errcodes\fP argument may be omitted from the argument list.
+
+.SH NOTES
+.ft R
+Completion of MPI_Comm_spawn in the parent does not necessarily mean that MPI_Init has been called in the children (although the returned intercommunicator can be used immediately).
+
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Comm_spawn_multiple(3)
+MPI_Comm_get_parent(3)
+mpirun(1)
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_spawn_multiple.3

@@ -0,0 +1,239 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2018 Cisco Systems, Inc.  All rights reserved
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_spawn_multiple 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_spawn_multiple\fP \- Spawns multiple binaries, or the same binary with multiple sets of arguments.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_spawn_multiple(int \fIcount\fP, char *\fIarray_of_commands\fP[],
+	char **\fIarray_of_argv\fP[], const int \fIarray_of_maxprocs\fP[], const MPI_Info
+	\fIarray_of_info\fP[], int \fIroot\fP, MPI_Comm \fIcomm\fP, MPI_Comm *\fIintercomm\fP,
+	int \fIarray_of_errcodes\fP[])
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+count
+Number of commands (positive integer, significant to MPI only at \fIroot\fP -- see NOTES).
+.TP 1i
+array_of_commands
+Programs to be executed (array of strings, significant only at \fIroot\fP).
+.TP 1i
+array_of_argv
+Arguments for \fIcommands\fP (array of array of strings,  significant only at \fIroot\fP).
+.TP 1i
+array_of_maxprocs
+Maximum number of processes to start for each command (array of integers, significant only at \fIroot\fP).
+.TP 1i
+array_of_info
+Info objects telling the runtime system where and how to start processes (array of handles, significant only at \fIroot\fP).
+.TP 1i
+root
+Rank of process in which previous arguments are examined (integer).
+.TP 1i
+comm
+Intracommunicator containing group of spawning processes (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+intercomm
+Intercommunicator between original group and the newly spawned group (handle).
+.TP 1i
+array_of_errcodes
+One code per process (array of integers).
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Comm_spawn_multiple is identical to MPI_Comm_spawn(3) except that
+it can specify multiple executables. The first argument, \fIcount\fP,
+indicates the number of executables. The next three arguments are
+arrays of the corresponding arguments in MPI_Comm_spawn(3). The next
+argument, \fIarray_of_info\fP, is an array of \fIinfo\fP arguments, one
+for each executable. See the INFO ARGUMENTS section for more information.
+.sp
+For the Fortran version of \fIarray_of_argv\fP, the element \fIarray_of_argv\fP(i,j) is the jth argument to command number i.
+.sp
+In any language, an application may use the constant MPI_ARGVS_NULL (which is likely to be (char ***)0 in C) to specify that no arguments should be passed to any commands. The effect of setting individual elements of \fIarray_of_argv\fP to MPI_ARGV_NULL is not defined. To specify arguments for some commands but not others, the commands without arguments should have a corresponding \fIargv\fP whose first element is null ((char *)0 in C and empty string in Fortran).
+.sp
+All of the spawned processes have the same MPI_COMM_WORLD. Their ranks in MPI_COMM_WORLD correspond directly to the order in which the commands are specified in MPI_Comm_spawn_multiple. Assume that m1 processes are generated by the first command, m2 by the second, etc. The processes corresponding to the first command have ranks 0, 1,..., m1-1. The processes in the second command have ranks m1, m1+1, ..., m1+m2-1. The processes in the third have ranks m1+m2, m1+m2+1, ..., m1+m2+m3-1, etc.
+.sp
+The \fIarray_of_errcodes\fP argument is 1-dimensional array of size
+.sp
+.nf
+	 _ count
+	\\       n ,
+	/_ i=1   i
+.fi
+.sp
+where i is the ith element of \fIarray_of_maxprocs\fP. Command number \fIi\fP corresponds to the i contiguous slots in this array from element
+.sp
+.nf
+                      _              _
+	 _ \fIi\fP-1          |   _ \fIi\fP          |
+	\\       n ,  to |  \\      n      | -1
+	/_ \fIj\fP=1   i      |  /_ \fIj\fP=1  j     |
+                     |_              _|
+.fi
+.sp
+Error codes are treated as for MPI_Comm_spawn(3).
+
+
+.SH INFO ARGUMENTS
+The following keys for \fIinfo\fP are recognized in "Open MPI". (The reserved values mentioned in Section 5.3.4 of the MPI-2 standard are not implemented.)
+.sp
+.sp
+.nf
+Key                    Type     Description
+---                    ----     -----------
+
+host                   char *   Comma-separated list of hosts on which
+                                the processes should be spawned.  See
+                                the \fIorte_host\fP man page for an
+                                explanation of how this will be used.
+hostfile               char *   Hostfile containing the hosts on which
+                                the processes are to be spawned. See
+                                the \fIorte_hostfile\fP man page for
+                                an explanation of how this will be
+                                used.
+add-host               char *   Add the specified hosts to the list of
+                                hosts known to this job and use it for
+                                the associated processes. This will be
+                                used similarly to the -host option.
+add-hostfile           char *   Hostfile containing hosts to be added
+                                to the list of hosts known to this job
+                                and use it for the associated
+                                process. This will be used similarly
+                                to the -hostfile option.
+wdir                   char *   Directory where the executable is
+                                located. If files are to be
+                                pre-positioned, then this location is
+                                the desired working directory at time
+                                of execution - if not specified, then
+                                it will automatically be set to
+                                \fIompi_preload_files_dest_dir\fP.
+ompi_prefix            char *   Same as the --prefix command line
+                                argument to mpirun.
+ompi_preload_binary    bool     If set to true, pre-position the
+                                specified executable onto the remote
+                                host. A destination directory must
+                                also be provided.
+ompi_preload_files     char *   A comma-separated list of files that
+                                are to be pre-positioned in addition
+                                to the executable.  Note that this
+                                option does not depend upon
+                                \fIompi_preload_binary\fP - files can
+                                be moved to the target even if an
+                                executable is not moved.
+ompi_stdin_target      char *   Comma-delimited list of ranks to
+                                receive stdin when forwarded.
+ompi_non_mpi           bool     If set to true, launching a non-MPI
+                                application; the returned communicator
+                                will be MPI_COMM_NULL. Failure to set
+                                this flag when launching a non-MPI
+                                application will cause both the child
+                                and parent jobs to "hang".
+ompi_param             char *   Pass an OMPI MCA parameter to the
+                                child job.  If that parameter already
+                                exists in the environment, the value
+                                will be overwritten by the provided
+                                value.
+mapper                 char *   Mapper to be used for this job
+map_by                 char *   Mapping directive indicating how
+                                processes are to be mapped (slot,
+                                node, socket, etc.).
+rank_by                char *   Ranking directive indicating how
+                                processes are to be ranked (slot,
+                                node, socket, etc.).
+bind_to                char *   Binding directive indicating how
+                                processes are to be bound (core, slot,
+                                node, socket, etc.).
+path                   char *   List of directories to search for
+                                the executable
+npernode               char *   Number of processes to spawn on
+                                each node of the allocation
+pernode                bool     Equivalent to npernode of 1
+ppr                    char *   Spawn specified number of processes
+                                on each of the identified object type
+env                    char *   Newline-delimited list of envars to
+                                be passed to the spawned procs
+.fi
+
+.sp
+\fIbool\fP info keys are actually strings but are evaluated as
+follows: if the string value is a number, it is converted to an
+integer and cast to a boolean (meaning that zero integers are false
+and non-zero values are true).  If the string value is
+(case-insensitive) "yes" or "true", the boolean is true.  If the
+string value is (case-insensitive) "no" or "false", the boolean is
+false.  All other string values are unrecognized, and therefore false.
+
+.sp
+Note that if any of the info handles have \fIompi_non_mpi\fP set to
+true, then all info handles must have it set to true.  If some are set
+to true, but others are set to false (or are unset), MPI_ERR_INFO will
+be returned.
+
+.sp
+Note that in "Open MPI", the first array location in \fIarray_of_info\fP is applied to all the commands in \fIarray_of_commands\fP.
+
+.SH NOTES
+The argument \fIcount\fP is interpreted by MPI only at the root, as is \fIarray_of_argv\fP. Since the leading dimension of \fIarray_of_argv\fP is \fIcount\fP, a nonpositive value of \fIcount\fP at a nonroot node could theoretically cause a runtime bounds check error, even though \fIarray_of_argv\fP should be ignored by the subroutine. If this happens, you should explicitly supply a reasonable value of \fIcount\fP on the nonroot nodes.
+.sp
+Similar to MPI_Comm_spawn(3), it is the application's responsibility
+to terminate each individual set of argv in the
+.I array_of_argv
+argument.  In C, each argv array is terminated by a NULL pointer.  In
+Fortran, each argv array is terminated by an empty string (note that
+compilers will not automatically insert this blank string; the
+application must ensure to have enough space for an empty string entry
+as the last element of the array).
+.sp
+Other restrictions apply to the
+.I array_of_argv
+parameter; see MPI_Comm_spawn(3)'s description of the
+.I argv
+parameter for more details.
+.sp
+MPI-3.1 implies (but does not directly state) that the argument
+\fIarray_of_commands\fP must be an array of strings of length
+\fIcount\fP.  Unlike the \fIarray_of_argv\fP parameter,
+\fIarray_of_commands\fP does not need to be terminated with a NULL
+pointer in C or a blank string in Fortran.  Older versions of Open MPI
+required that \fIarray_of_commands\fP be terminated with a blank
+string in Fortran; that is no longer required in this version of Open
+MPI.
+.sp
+Calling MPI_Comm_spawn(3) many times would create many sets of
+children with different MPI_COMM_WORLDs, whereas
+MPI_Comm_spawn_multiple creates children with a single MPI_COMM_WORLD,
+so the two methods are not completely equivalent. Also if you need to
+spawn multiple executables, you may get better performance by using
+MPI_Comm_spawn_multiple instead of calling MPI_Comm_spawn(3) several
+times.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Comm_spawn(3)
+MPI_Comm_get_parent(3)
+mpirun(1)

macx64/mpi/openmpi/share/man/man3/MPI_Comm_split.3

@@ -0,0 +1,82 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_split 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_split \fP \- Creates new communicators based on colors and keys.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_split(MPI_Comm \fIcomm\fP, int\fI color\fP, int\fI key\fP,
+	MPI_Comm *\fInewcomm\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+.TP 1i
+color
+Control of subset assignment (nonnegative integer).
+.TP 1i
+key
+Control of rank assignment (integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+New communicator (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function partitions the group associated with comm into disjoint subgroups, one for each value of color. Each subgroup contains all processes of the same color. Within each subgroup, the processes are ranked in the order defined by the value of the argument key, with ties broken according to their rank in the old group. A new communicator is created for each subgroup and returned in newcomm. A process may supply the color value MPI_UNDEFINED, in which case newcomm returns MPI_COMM_NULL. This is a collective call, but each process is permitted to provide different values for color and key.
+.sp
+When you call MPI_Comm_split on an inter-communicator, the processes on the left with the same color as those on the right combine to create a new inter-communicator.  The key argument describes the relative rank of processes on each side of the inter-communicator.  The function returns MPI_COMM_NULL for  those colors that are specified on only one side of the inter-communicator, or for those that specify MPI_UNDEFINED as the color.
+.sp
+A call to MPI_Comm_create(\fIcomm\fP, \fIgroup\fP, \fInewcomm\fP) is equivalent to a call to MPI_Comm_split(\fIcomm\fP, \fIcolor\fP,\fI key\fP, \fInewcomm\fP), where all members of \fIgroup\fP provide \fIcolor\fP = 0 and \fIkey\fP = rank in group, and all processes that are not members of \fIgroup\fP provide \fIcolor\fP = MPI_UNDEFINED. The function MPI_Comm_split allows more general partitioning of a group into one or more subgroups with optional reordering.
+.sp
+The value of \fIcolor\fP must be nonnegative or MPI_UNDEFINED.
+
+.SH NOTES
+.ft R
+This is an extremely powerful mechanism for
+dividing a single communicating group of processes into k subgroups, with k
+chosen implicitly by the user (by the number of colors asserted over all
+the processes). Each resulting communicator will be nonoverlapping. Such a division could be useful for defining a hierarchy of computations, such as for multigrid or linear algebra.
+.sp
+Multiple calls to MPI_Comm_split can be used to overcome the requirement that any call have no overlap of the resulting communicators (each process is of only one color per call). In this way, multiple overlapping communication structures can be created. Creative use of the color and key in such splitting operations is encouraged.
+.sp
+Note that, for a fixed color, the keys need not be unique. It is MPI_Comm_split's responsibility to sort processes in ascending order according to this key, and to break ties in a consistent way. If all the keys are specified in the same way, then all the processes in a given color will have the relative rank order as they did in their parent group. (In general, they will have different ranks.)
+.sp
+Essentially, making the key value zero for all processes of a given color
+means that one needn't really pay attention to the rank-order of the processes in the new communicator.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Comm_create
+.br
+MPI_Intercomm_create
+.br
+MPI_Comm_dup
+.br
+MPI_Comm_free
+
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_split_type.3

@@ -0,0 +1,146 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright (c) 2010-2015 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_split_type 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_split_type \fP \- Creates new communicators based on colors and keys.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_split_type(MPI_Comm \fIcomm\fP, int\fI split_type\fP, int\fI key\fP,
+	MPI_Info info, MPI_Comm *\fInewcomm\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+.TP 1i
+split_type
+Type of processes to be grouped together (integer).
+.TP 1i
+key
+Control of rank assignment (integer).
+.TP 1i
+info
+Info argument (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+newcomm
+New communicator (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function partitions the group associated with \fIcomm\fP into disjoint subgroups, based on
+the type specied by \fIsplit_type\fP. Each subgroup contains all processes of the same type.
+Within each subgroup, the processes are ranked in the order defined by the value of the
+argument \fIkey\fP, with ties broken according to their rank in the old group. A new communicator
+is created for each subgroup and returned in newcomm. This is a collective call;
+all processes must provide the same \fIsplit_type\fP, but each process is permitted to provide
+different values for key. An exception to this rule is that a process may supply the type
+value MPI_UNDEFINED, in which case newcomm returns MPI_COMM_NULL.
+
+.SH SPLIT TYPES
+.ft R
+.TP 1i
+MPI_COMM_TYPE_SHARED
+This type splits the communicator into subcommunicators, each of which can create a shared memory region.
+
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_NODE
+Synonym for MPI_COMM_TYPE_SHARED.
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_HWTHREAD
+This type splits the communicator into subcommunicators, each of which belongs to the same hardware thread.
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_CORE
+This type splits the communicator into subcommunicators, each of which belongs to the same core/processing unit.
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_L1CACHE
+This type splits the communicator into subcommunicators, each of which belongs to the same L1 cache.
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_L2CACHE
+This type splits the communicator into subcommunicators, each of which belongs to the same L2 cache.
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_L3CACHE
+This type splits the communicator into subcommunicators, each of which belongs to the same L3 cache.
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_SOCKET
+This type splits the communicator into subcommunicators, each of which belongs to the same socket.
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_NUMA
+This type splits the communicator into subcommunicators, each of which belongs to the same NUMA-node.
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_BOARD
+This type splits the communicator into subcommunicators, each of which belongs to the same board.
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_HOST
+This type splits the communicator into subcommunicators, each of which belongs to the same host.
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_CU
+This type splits the communicator into subcommunicators, each of which belongs to the same computational unit.
+.ft R
+.TP 1i
+OMPI_COMM_TYPE_CLUSTER
+This type splits the communicator into subcommunicators, each of which belongs to the same cluster.
+
+.SH NOTES
+.sp
+The communicator keys denoted with an
+.I OMPI_
+prefix instead of an
+.I MPI_
+prefix are specific to Open MPI, and are not part of the MPI
+standard.  Their use should be protected by the
+.I OPEN_MPI
+C preprocessor macro.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value
+of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for
+I/O function errors. The error handler may be changed with
+MPI_Comm_set_errhandler; the predefined error handler
+MPI_ERRORS_RETURN may be used to cause error values to be
+returned. Note that MPI does not guarantee that an MPI program can
+continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Comm_create
+.br
+MPI_Intercomm_create
+.br
+MPI_Comm_dup
+.br
+MPI_Comm_free
+.br
+MPI_Comm_split
+

macx64/mpi/openmpi/share/man/man3/MPI_Comm_test_inter.3

@@ -0,0 +1,68 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Comm_test_inter 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Comm_test_inter \fP \- Tests to see if a comm is an intercommunicator.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Comm_test_inter(MPI_Comm \fIcomm\fP, int\fI *flag\fP)
+
+.fi
+.SH INPUT PARAMETER
+.ft R
+.TP 1i
+comm
+Communicator (handle).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+flag     (Logical.)
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This local routine allows the calling process to determine the type of a communicator. It returns true for an intercommunicator, false for an intracommunicator.
+.sp
+The type of communicator also affects the value returned by three other functions.  When dealing with an intracommunicator (enables communication within a single group), the functions listed below return the expected values, group size, group, and rank.  When dealing with an inter-communicator, however, they return the following values:
+.sp
+.nf
+MPI_Comm_size	Returns the size of the local group.
+MPI_Comm_group	Returns the local group.
+MPI_Comm_rank	Returns the rank in the local group.
+.fi
+.sp
+To return the remote group and remote group size of an inter-communicator, use the MPI_Comm_remote_group and MPI_Comm_remote_size functions.
+.sp
+The operation MPI_Comm_compare is valid for intercommunicators. Both communicators must be either intra- or intercommunicators, or else MPI_UNEQUAL results. Both corresponding local and remote groups must compare correctly to get the results MPI_CONGRUENT and MPI_SIMILAR. In particular, it is possible for MPI_SIMILAR to result because either the local or remote groups were similar but not identical.
+.sp
+The following accessors provide consistent access to the remote group of an
+intercommunicator: MPI_Comm_remote_size, MPI_Comm_remote_group.
+.sp
+The intercommunicator accessors (MPI_Comm_test_inter, MPI_Comm_remote_size, MPI_Comm_remote_group) are all local operations.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+.nf
+MPI_Comm_remote_group
+MPI_Comm_remote_size
+MPI_Intercomm_create
+MPI_Intercomm_merge
+

macx64/mpi/openmpi/share/man/man3/MPI_Compare_and_swap.3

@@ -0,0 +1,87 @@
+.\" -*- nroff -*-
+.\" Copyright 2013-2015 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Compare_and_swap 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Compare_and_swap\fP \- Perform RMA compare-and-swap
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Compare_and_swap(const void *\fIorigin_addr\fP, const void *\fIcompare_addr\fP,
+	void *\fIresult_addr\fP, MPI_Datatype \fIdatatype\fP, int \fItarget_rank\fP,
+	MPI_Aint \fItarget_disp\fP, MPI_Win \fIwin\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+origin_addr
+Initial address of buffer (choice).
+.ft R
+.TP
+compare_addr
+Initial address of compare buffer (choice).
+.ft R
+.TP
+result_addr
+Initial address of result buffer (choice).
+.ft R
+.TP
+datatype
+Data type of the entry in origin, result, and target buffers (handle).
+.ft R
+.TP 1i
+target_rank
+Rank of target (nonnegative integer).
+.ft R
+.TP 1i
+target_disp
+Displacement from start of window to beginning of target buffer (nonnegative integer).
+.ft R
+.TP 1i
+win
+Window object (handle).
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+This function compares one element of type \fIdatatype\fP in the compare buffer \fIcompare_addr\fP with the buffer at offset \fItarget_disp\fP in the target window specified by \fItarget_rank\fP and \fIwin\fP and replaces the value at the target with the value in the origin buffer \fIorigin_addr\fP if the compare buffer and the target buffer are identical. The original value at the target is returned in the buffer \fIresult_addr\fP. The parameter \fIdatatype\fP must belong to one of the following categories of predefined datatypes: C integer, Fortran integer, Logical, Multi-language types, or Byte as specified in MPI-3 § 5.9.2 on page 176.
+.sp
+The origin and result buffers (\fIorigin_addr\fP and \fIresult_addr\fP) must be disjoint.
+
+.SH FORTRAN 77 NOTES
+.ft R
+The MPI standard prescribes portable Fortran syntax for
+the \fITARGET_DISP\fP argument only for Fortran 90.  FORTRAN 77
+users may use the non-portable syntax
+.sp
+.nf
+     INTEGER*MPI_ADDRESS_KIND \fITARGET_DISP\fP
+.fi
+.sp
+where MPI_ADDRESS_KIND is a constant defined in mpif.h
+and gives the length of the declared integer in bytes.
+
+.SH NOTES
+It is the user's responsibility to guarantee that, when
+using the accumulate functions, the target displacement argument is such
+that accesses to the window are properly aligned according to the data
+type arguments in the call to the \fBMPI_Compare_and_swap\fP function.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler
+may be changed with \fBMPI_Comm_set_errhandler\fP; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.

macx64/mpi/openmpi/share/man/man3/MPI_Dims_create.3

@@ -0,0 +1,68 @@
+.\" -*- nroff -*-
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Dims_create 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Dims_create \fP \- Creates a division of processors in a Cartesian grid.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Dims_create(int \fInnodes\fP, int\fI ndims\fP, int\fI dims\fP[])
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+nnodes
+Number of nodes in a grid (integer).
+.TP 1i
+ndims
+Number of Cartesian dimensions (integer).
+
+.SH IN/OUT PARAMETER
+.TP 1i
+dims
+Integer array of size ndims specifying the number of nodes in each dimension.
+
+.SH OUTPUT PARAMETER
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+For Cartesian topologies, the function MPI_Dims_create helps the user select a balanced distribution of processes per coordinate direction, depending on the number of processes in the group to be balanced and optional constraints that can be specified by the user. One use is to partition all the processes (the size of MPI_COMM_WORLD's group) into an n-dimensional topology.
+.sp
+The entries in the array \fIdims\fP are set to describe a Cartesian grid with \fIndims\fP dimensions and a total of \fInnodes\fP nodes. The dimensions are set to be as close to each other as possible, using an appropriate divisibility algorithm. The caller may further constrain the operation of this routine by specifying elements of array dims. If dims[i] is set to a positive number, the routine will not modify the number of nodes in dimension i; only those entries where  dims[i] = 0 are modified by the call.
+.sp
+Negative input values of dims[i] are erroneous. An error will occur if
+nnodes is not a multiple of ((pi) over (i, dims[i] != 0)) dims[i].
+.sp
+For dims[i] set by the call, dims[i] will be ordered in nonincreasing order. Array dims is suitable for use as input to routine MPI_Cart_create. MPI_Dims_create is local.
+.sp
+\fBExample:\fP
+.nf
+
+dims
+before					dims
+call		function call		on return
+-----------------------------------------------------
+(0,0)	MPI_Dims_create(6, 2, dims)	(3,2)
+(0,0)	MPI_Dims_create(7, 2, dims) 	(7,1)
+(0,3,0)	MPI_Dims_create(6, 3, dims)	(2,3,1)
+(0,3,0)	MPI_Dims_create(7, 3, dims)	erroneous call
+------------------------------------------------------
+
+.fi
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument. C++ functions do not return errors. If the default error handler is set to MPI::ERRORS_THROW_EXCEPTIONS, then on error the C++ exception mechanism will be used to throw an MPI::Exception object.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+

macx64/mpi/openmpi/share/man/man3/MPI_Dist_graph_create.3

@@ -0,0 +1,112 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Dist_graph_create 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Dist_graph_create \fP \- Makes a new communicator to which topology information has been attached.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Dist_graph_create(MPI_Comm \fIcomm_old\fP, int\fI n\fP, const int\fI sources[]\fP,
+	const int\fI degrees[]\fP, const int\fI destinations\fP[], const int\fI weights\fP[],
+        MPI_Info info, int\fI reorder\fP, MPI_Comm\fI *comm_dist_graph\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm_old
+Input communicator without topology (handle).
+.TP 1i
+n
+Number of source nodes for which this process specifies edges (non-negative integer).
+.TP 1i
+sources
+Array containing the \fIn\fP source nodes for which this process species edges (array of non-negative integers).
+.TP 1i
+degrees
+Array specifying the number of destinations for each source node in the source node array (array of non-negative integers).
+.TP 1i
+destinations
+Destination nodes for the source nodes in the source node array (array of non-negative integers).
+.TP 1i
+weights
+Weights for source to destination edges (array of non-negative integers).
+.TP 1i
+Hints on optimization and interpretation of weights (handle).
+.TP 1i
+reorder
+Ranking may be reordered (true) or not (false) (logical).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+comm_dist_graph
+Communicator with distibuted graph topology added (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Dist_graph_create creates a new communicator \fIcomm_dist_graph\fP with distrubuted
+graph topology and returns a handle to the new communicator. The number of processes in
+\fIcomm_dist_graph\fP is identical to the number of processes in \fIcomm_old\fP. Concretely, each process calls the
+constructor with a set of directed (source,destination) communication edges as described below.
+Every process passes an array of \fIn\fP source nodes in the \fIsources\fP array. For each source node, a
+non-negative number of destination nodes is specied in the \fIdegrees\fP array. The destination
+nodes are stored in the corresponding consecutive segment of the \fIdestinations\fP array. More
+precisely, if the i-th node in sources is s, this species \fIdegrees\fP[i] \fIedges\fP (s,d) with d of the j-th
+such edge stored in \fIdestinations\fP[\fIdegrees\fP[0]+...+\fIdegrees\fP[i-1]+j]. The weight of this edge is
+stored in \fIweights\fP[\fIdegrees\fP[0]+...+\fIdegrees\fP[i-1]+j]. Both the \fIsources\fP and the \fIdestinations\fP arrays
+may contain the same node more than once, and the order in which nodes are listed as
+destinations or sources is not signicant. Similarly, different processes may specify edges
+with the same source and destination nodes. Source and destination nodes must be process
+ranks of comm_old. Different processes may specify different numbers of source and
+destination nodes, as well as different source to destination edges. This allows a fully distributed
+specification of the communication graph. Isolated processes (i.e., processes with
+no outgoing or incoming edges, that is, processes that do not occur as source or destination
+node in the graph specication) are allowed. The call to MPI_Dist_graph_create is collective.
+
+If reorder = false, all processes will have the same rank in comm_dist_graph as in
+comm_old. If reorder = true then the MPI library is free to remap to other processes (of
+comm_old) in order to improve communication on the edges of the communication graph.
+The weight associated with each edge is a hint to the MPI library about the amount or
+intensity of communication on that edge, and may be used to compute a \"best\" reordering.
+
+.SH WEIGHTS
+.ft R
+Weights are specied as non-negative integers and can be used to influence the process
+remapping strategy and other internal MPI optimizations. For instance, approximate count
+arguments of later communication calls along specic edges could be used as their edge
+weights. Multiplicity of edges can likewise indicate more intense communication between
+pairs of processes. However, the exact meaning of edge weights is not specied by the MPI
+standard and is left to the implementation. An application can supply the special value
+MPI_UNWEIGHTED for the weight array to indicate that all edges have the same (effectively no)
+weight. It is erroneous to supply MPI_UNWEIGHTED for some but not
+all processes of comm_old. If the graph is weighted but \fIn\fP = 0, then MPI_WEIGHTS_EMPTY
+or any arbitrary array may be passed to weights. Note that MPI_UNWEIGHTED and
+MPI_WEIGHTS_EMPTY are not special weight values; rather they are special values for the
+total array argument. In Fortran, MPI_UNWEIGHTED and MPI_WEIGHTS_EMPTY are objects
+like MPI_BOTTOM (not usable for initialization or assignment). See MPI-3 § 2.5.4.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Dist_graph_create_adjacent
+MPI_Dist_graph_neighbors
+MPI_Dist_graph_neighbors_count
+

macx64/mpi/openmpi/share/man/man3/MPI_Dist_graph_create_adjacent.3

@@ -0,0 +1,106 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Dist_graph_create_adjacent 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Dist_graph_create_adjacent \fP \- Makes a new communicator to which topology information has been attached.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Dist_graph_create_adjacent(MPI_Comm \fIcomm_old\fP, int\fI indegree\fP, const int\fI sources[]\fP,
+	const int\fI sourceweights[]\fP, int\fI outdegree\fP, const int\fI destinations\fP[], const int\fI destweights\fP[],
+        MPI_Info info, int\fI reorder\fP, MPI_Comm\fI *comm_dist_graph\fP)
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm_old
+Input communicator without topology (handle).
+.TP 1i
+indegree
+Size of \fIsources\fP and \fIsourceweights\fP arrays (non-negative integer).
+.TP 1i
+sources
+Ranks of processes for which the calling process is a destination (array of non-negative integers).
+.TP 1i
+sourceweights
+Weights of the edges into the calling process (array of non-negative integers).
+.TP 1i
+outdegree
+Size of \fIdestinations\fP and \fIdestweights\fP arrays (non-negative integer).
+.TP 1i
+destinations
+Ranks of processes for which the calling process is a source (array of non-negative integers).
+.TP 1i
+destweights
+Weights of the edges out of the calling process (array of non-negative integers).
+.TP 1i
+Hints on optimization and interpretation of weights (handle).
+.TP 1i
+reorder
+Ranking may be reordered (true) or not (false) (logical).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+comm_dist_graph
+Communicator with distibuted graph topology added (handle).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Dist_graph_create_adjacent creats a new communicator \fIcomm_dist_graph\fP with distrubuted
+graph topology and returns a handle to the new communicator. The number of processes in
+\fIcomm_dist_graph\fP is identical to the number of processes in \fIcomm_old\fP. Each process passes all
+information about its incoming and outgoing edges in the virtual distributed graph topology.
+The calling processes must ensure that each edge of the graph is described in the source
+and in the destination process with the same weights. If there are multiple edges for a given
+(source,dest) pair, then the sequence of the weights of these edges does not matter. The
+complete communication topology is the combination of all edges shown in the \fIsources\fP arrays
+of all processes in comm_old, which must be identical to the combination of all edges shown
+in the \fIdestinations\fP arrays. Source and destination ranks must be process ranks of comm_old.
+This allows a fully distributed specication of the communication graph. Isolated processes
+(i.e., processes with no outgoing or incoming edges, that is, processes that have specied
+indegree and outdegree as zero and thus do not occur as source or destination rank in the
+graph specication) are allowed. The call to MPI_Dist_graph_create_adjacent is collective.
+
+.SH WEIGHTS
+.ft R
+Weights are specied as non-negative integers and can be used to influence the process
+remapping strategy and other internal MPI optimizations. For instance, approximate count
+arguments of later communication calls along specic edges could be used as their edge
+weights. Multiplicity of edges can likewise indicate more intense communication between
+pairs of processes. However, the exact meaning of edge weights is not specied by the MPI
+standard and is left to the implementation. An application can supply the special value
+MPI_UNWEIGHTED for the weight array to indicate that all edges have the same (effectively
+no) weight. It is erroneous to supply MPI_UNWEIGHTED for some but not all processes of
+comm_old. If the graph is weighted but \fIindegree\fP or \fIoutdegree\fP is zero, then
+MPI_WEIGHTS_EMPTY or any arbitrary array may be passed to sourceweights or destweights
+respectively. Note that MPI_UNWEIGHTED and MPI_WEIGHTS_EMPTY are not special weight values;
+rather they are special values for the total array argument. In Fortran, MPI_UNWEIGHTED
+and MPI_WEIGHTS_EMPTY are objects like MPI_BOTTOM (not usable for initialization or
+assignment). See MPI-3 § 2.5.4.
+
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Dist_graph_create
+MPI_Dist_graph_neighbors
+MPI_Dist_graph_neighbors_count
+

macx64/mpi/openmpi/share/man/man3/MPI_Dist_graph_neighbors.3

@@ -0,0 +1,73 @@
+.\" -*- nroff -*-
+.\" Copyright 2013 Los Alamos National Security, LLC. All rights reserved.
+.\" Copyright 2010 Cisco Systems, Inc.  All rights reserved.
+.\" Copyright 2006-2008 Sun Microsystems, Inc.
+.\" Copyright (c) 1996 Thinking Machines Corporation
+.\" $COPYRIGHT$
+.TH MPI_Dist_graph_neighbors 3 "Mar 26, 2019" "4.0.1" "Open MPI"
+.SH NAME
+\fBMPI_Dist_graph_neighbors \fP \- Returns the neighbors of the calling process in a distributed graph topology.
+
+.SH SYNTAX
+.ft R
+.SH C Syntax
+.nf
+#include <mpi.h>
+int MPI_Dist_graph_neighbors(MPI_Comm \fIcomm\fP, int \fImaxindegree\fP, int \fIsources\fP[], int \fIsourceweights\fP[],
+                             int \fImaxoutdegree\fP, int \fIdestinations\fP[], int \fIdestweights\fP[])
+
+.fi
+.SH INPUT PARAMETERS
+.ft R
+.TP 1i
+comm
+Communicator with distributed graph topology (handle).
+.TP 1i
+maxindegree
+Size of \fIsources\fP and \fIsourceweights\fP arrays (non-negative integer).
+.TP 1i
+maxoutdegree
+Size of \fIdestinations\fP and \fIdestweights\fP arrays (non-negative integer).
+
+.SH OUTPUT PARAMETERS
+.ft R
+.TP 1i
+sources
+Processes for which the calling process is a destination (array of non-negative integers).
+.TP 1i
+sourceweights
+Weights of the edges into the calling process (array of non-negative integers).
+.TP 1i
+destinations
+Processes for which the calling process is a source (array of non-negative integers).
+.TP 1i
+destweights
+Weights of the edges out of the calling process (array of non-negative integers).
+.ft R
+.TP 1i
+IERROR
+Fortran only: Error status (integer).
+
+.SH DESCRIPTION
+.ft R
+MPI_Dist_graph_neighbors returns the source and destination ranks in a distributed graph topology
+for the calling process. This call will return up to \fImaxindegree\fP source ranks in the \fIsources\fP array
+and up to \fImaxoutdegree\fP destination ranks in the \fIdestinations\fP array. If weights were
+specified at the time of the communicator's creation then the associated weights
+are returned in the \fIsourceweights\fP and \fI destweights\fP arrays. If the communicator
+was created with MPI_Dist_graph_create_adjacent then the order of the values in \fIsources\fP and
+\fIdestinations\fP is identical to the input that was used by the process with the same rank in
+comm_old in the creation call.
+
+.fi
+.SH ERRORS
+Almost all MPI routines return an error value; C routines as the value of the function and Fortran routines in the last argument.
+.sp
+Before the error value is returned, the current MPI error handler is
+called. By default, this error handler aborts the MPI job, except for I/O function errors. The error handler may be changed with MPI_Comm_set_errhandler; the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error.
+
+.SH SEE ALSO
+.ft R
+.sp
+MPI_Dist_graph_neighbors_count
+