Overlapping Communication with Computation

• The MPI programs we developed so far used blocking send and receive operations whenever they needed to perform point-to-point communication. Recall that a blocking send operation remains blocked until the message has been copied out of the send buffer (either into a system buffer at the source process or sent to the destination process).
• Similarly, a blocking receive operation returns only after the message has been received and copied into the receive buffer.
• It will be preferable if we can overlap the transmission of the data with the computation, as many recent distributed-memory parallel computers have dedicated communication controllers that can perform the transmission of messages without interrupting the CPUs.