Overlapping Communication with Computation

• The MPI programs we developed so far used blocking send and receive operations whenever they needed to perform point-to-point communication.
• Recall that a blocking send operation remains blocked until the message has been copied out of the send buffer
  • either into a system buffer at the source process
  • or sent to the destination process.
• Similarly, a blocking receive operation returns only after the message has been received and copied into the receive buffer.
• It will be preferable if we can overlap the transmission of the data with the computation.
• Since many recent distributed-memory parallel computers have dedicated communication controllers,
• that can perform the transmission of messages without interrupting the CPUs.