Non-Blocking Message Passing Operations

• In blocking protocols, the overhead of guaranteeing semantic correctness was paid in the form of idling (non-buffered) or buffer management (buffered).
• Often, it is possible to require the programmer to ensure semantic correctness and provide a fast send/receive operation that incurs little overhead.
• This class of non-blocking protocols returns from the send or receive operation before it is semantically safe to do so. Consequently, the user must be careful not to alter data that may be potentially participating in a communication operation.
• Non-blocking operations are generally accompanied by a check-status operation, which indicates whether the semantics of a previously initiated transfer may be violated or not.
• Upon return from a non-blocking send or receive operation, the process is free to perform any computation that does not depend upon the completion of the operation. Later in the program, the process can check whether or not the non-blocking operation has completed, and, if necessary, wait for its completion.

  Figure 3: Space of possible protocols for send and receive operations.

  
• As illustrated in Fig. 3, non-blocking operations can themselves be buffered or non-buffered.
• In the non-buffered case, a process wishing to send data to another simply posts a pending message and returns to the user program. The program can then do other useful work. At some point in the future, when the corresponding receive is posted, the communication operation is initiated.
• When this operation is completed, the check-status operation indicates that it is safe for the programmer to touch this data. This transfer is indicated in Fig. 4a.

  Figure 4: Non-blocking non-buffered send and receive operations (a) in absence of communication hardware; (b) in presence of communication hardware.

  
• Comparing Figures 4a and 1a, it is easy to see that the idling time when the process is waiting for the corresponding receive in a blocking operation can now be utilized for computation, provided it does not update the data being sent.
• This alleviates the major bottleneck associated with the former at the expense of some program restructuring. The benefits of non-blocking operations are further enhanced by the presence of dedicated communication hardware.
• This is illustrated in Fig. 4b. In this case, the communication overhead can be almost entirely masked by non-blocking operations. In this case, however, the data being received is unsafe for the duration of the receive operation.
• Non-blocking operations can also be used with a buffered protocol. In this case, the sender initiates a DMA operation and returns immediately.
• The data becomes safe the moment the DMA operation has been completed. At the receiving end, the receive operation initiates a transfer from the sender's buffer to the receiver's target location. Using buffers with non-blocking operation has the effect of reducing the time during which the data is unsafe.
• Typical message-passing libraries such as Message Passing Interface (MPI) and Parallel Virtual Machine (PVM) implement both blocking and non-blocking operations.
  • Blocking operations facilitate safe and easier programming and non-blocking operations are useful for performance optimization by masking communication overhead.
  • One must, however, be careful using non-blocking protocols since errors can result from unsafe access to data that is in the process of being communicated.