SIMD Architecture

• The SIMD model of parallel computing consists of two parts: a front-end computer of the usual von Neumann style, and a processor array as shown in Fig. 1.2c.
• Each processor in the array has a small amount of local memory where the distributed data resides while it is being processed in parallel.
• The processor array is connected to the memory bus of the front end so that the front end can randomly access the local processor memories as if it were another memory.
• The application program is executed by the front end in the usual serial way, but issues commands to the processor array to carry out SIMD operations in parallel.
• The similarity between serial and data parallel programming is one of the strong points of data parallelism.
• Synchronization is made irrelevant by the lock-step synchronization of the processors. Processors either do nothing or exactly the same operations at the same time.
• In SIMD architecture, parallelism is exploited by applying simultaneous operations across large sets of data.

  Figure 1.5: Two SIMD Schemes.
• There are two main configurations that have been used in SIMD machines (see Fig. 1.5).
  1. In the first scheme, each processor has its own local memory. Processors can communicate with each other through the interconnection network.
     • If the interconnection network does not provide direct connection between a given pair of processors, then this pair can exchange data via an intermediate processor.
     • The ILLIAC IV used such an interconnection scheme. The interconnection network in the ILLIAC IV allowed each processor to communicate directly with four neighboring processors in an $8 \times 8$ matrix pattern such that the i th processor can communicate directly with the $(i-1)^{th},(i+1)^{th},(i-8)^{th},and (i+8)^{th}$ processors.
  2. In the second SIMD scheme, processors and memory modules communicate with each other via the interconnection network.
     • Two processors can transfer data between each other via intermediate memory module(s) or possibly via intermediate processor(s). The BSP (Burroughs' Scientific Processor) used the second SIMD scheme.