Message Passing vs Shared Memory

• Shared memory enjoys the desirable feature that all communications are done using implicit loads and stores to a global address space.
• Another fundamental feature of shared memory is that synchronization and communication are distinct. Special synchronization operations (mechanisms), in addition to the loads and stores operations, need to be employed in order to detect when data have been produced and/or consumed.
• On the other hand, message passing employs an explicit communication model. Explicit messages are exchanged among processors.
• Synchronization and communication are unified in message passing. The generation of remote, asynchronous events is an integral part of the message passing communication model.
• It is important, however, to indicate that shared memory and message passing communication models are universal; that is, it is possible to employ one to simulate the other.
  • However, it is observed that it is easier to simulate shared memory using message passing than the converse.
  • This is basically because of the asynchronous event semantics of message passing as compared to the polling semantics of the shared memory.
• The shared memory communication model allows the programmer to concentrate on the issues related to parallelism by relieving him/her of the details of the interprocessor communication.
• In that sense, the shared memory communication model represents a straightforward extension of the uniprocessor programming paradigm. In addition, shared memory semantics are independent of the physical location and therefore they are open to the dynamic optimization offered by the underlying operating system.
• On the other hand, the shared memory communication model is in essence a polling interface. This is a drawback as far as synchronization is concerned.
• Message passing can be characterized as employing an interrupt-driven communication model. In message passing, messages include both data and synchronization in a single unit. As such, the message passing communication model lends itself to those operating system activities in which communication patterns are explicitly known in advance, for example, I/O, interprocessor interrupts, and task and data migration.
• On the other hand, message passing suffers from the need for marshaling cost, that is, the cost of assembling and disassembling of the message.
• One natural conclusion arising from the above discussion is that shared memory and message passing communication models each lend themselves naturally to certain application domains. Shared memory manifests itself to application writers while message passing manifests itself to operating systems designers.
• It is therefore natural to consider combining both shared memory and message passing in general-purpose multiprocessor systems. This has been the main driving force behind systems such as the Stanford FLexible Architecture for SHared memory (FLASH) system. It is a multiprocessor system that efficiently integrates support for shared memory and message passing while minimizing both hardware and software overhead.