All-to-All

• Alltoall; the all-to-all communication operation is performed in MPI by using the MPI_Alltoall function.

  int MPI_Alltoall(void *sendbuf, int sendcount, 
          MPI_Datatype senddatatype, void *recvbuf, int recvcount, 
          MPI_Datatype recvdatatype, MPI_Comm comm)
  

• Each process sends a different portion of the sendbuf array to each other process, including itself.
• Each process sends to process i sendcount contiguous elements of type senddatatype starting from the i * sendcount location of its sendbuf array.
• The data that are received are stored in the recvbuf array.
• Each process receives from process i recvcount elements of type recvdatatype and stores them in its recvbuf array starting at location i * recvcount.
• MPI also provides a vector variant of the all-to-all personalized communication operation called MPI_Alltoallv that allows different amounts of data to be sent.

Figure 6: Diagram for Alltoall.