Parallel Processing Systems

Traditional multiprocessor system (share a common bus, clock, and memory), tightly-coupled; multiprocessing. The desire for increased throughput has led to system designs in which multiple streams of processing occurs in parallel.

• Tightly coupled system processors share memory and a clock; communication usually takes place through the shared memory. For a system with n processors that is to run n or more separate programs, the speedup may approach n. It will not reach n because there will be some contention for access to shared elements such as the memory system.
• Advantages of parallelism:
  • higher throughput and better fault tolerance
  • Economical (?)
  • Increased availability ($\neq$ reliability)
• Symmetric multiprocessing (SMP), a symmetric multi-processor system shares the execution of the operating system amongst all of the processors - it is usually multi-threaded and contains no block structures. The CPUs are equal i.e. we say that they are all peers.
  • Each processor runs identical copy of OS
  • Many processes at once without performance loss
  • Most modern operating systems support SMP
• Asymmetric multiprocessing, an asymmetric multi-processor system contains a single CPU called the master that carefully controls access to single threaded sections of the kernel - this processor controls the activities of the slave processors. Asymmetric MP systems are less efficient.
  • Each processor is assigned specific task; master processor schedules, allocates work to slave processors.
  • Mostly for specialized high-end computation
• What can we say about an n-processor system that has m $<$ n application programs to run? Unless some of the application programs can support multiple threads of simultaneous execution, then the speedup may only approach m (since n - m processors are idle).
• As programmers, you will learn about thread models (and support libraries) that allow you to develop multi-threaded applications. Important applications of multi-threading include database servers, and users of databases will be aware that aspects of simultaneous access can require special care.
• In specialized application areas such as high speed digital signal processing (DSP) e.g. radar processing, digital mobile base station processing, etc., hybrid parallel systems may be assembled with individual CPUs having significant private memory plus a connection onto a common shared memory.
• While these tightly coupled systems require specialized hardware support in order that the CPUs can share the common memory system, another approach is to use a network to join together more conventional systems into what is termed a distributed system.

Figure 1.4: SMP architecture, Client-Server