List of Figures

1. View of the Field and Abstraction Layers
2. MIMD Shared Memory, MIMD Distributed Memory, SIMD Distributed Computers, and Clusters.
3. Interconnection Network Taxonomy and Four Decades of Computing.
4. SISD, SIMD, amd MIMD Architectures.
5. Two SIMD Schemes.
6. Two MIMD Categories; Shared Memory and Message Passing MIMD Architectures.
7. Bus Based and Switch Based Shared Memory INs, Single Bus Based and Multiple Bus Based Shared Memory INs.
8. Examples of Static Topologies.
9. Single-Stage, Multi-Stage and Crossbar Switch Dynamics INs.
10. Performance Comparisons of Some Dynamics INs and Performance Characteristics of Static INs.
11. Cooperative-Communicating with other processes.
12. One sided-Communicating with other processes.
13. Single Bus System.
14. Multiple bus with full bus memory connection (MBFBMC); multiple bus with single bus-memory connection (MBSBMC); multiple bus with partial bus memory connection (MBPBMC); and multiple bus with class-based memory connection (MBCBMC).
15. An $8 \times 8$ crossbar network (a) straight switch setting; and (b) diagonal switch setting.
16. The different settings of the $2 \times 2$ SE.
17. Multistage interconnection network and an example $8 \times 8$ Shuffle-Exchange network (SEN).
18. Multistage interconnection network and an example $8 \times 8$ Shuffle-Exchange network (SEN).
19. Completely Connected Network.
20. A 4-cube.
21. A $3 \times 3 \times 2$ mesh network.
22. Performance Comparison of Dynamic Networks.
23. Performance Characteristics of Static Networks.
24. MPI messages.
25. Data+Envelope.
26. MPI basic datatypes for C.
27. left-top; Correct Execution of Library Calls, right-top; Incorrect Execution of Library Calls, left-bottom; Correct Execution of Library Calls with Pending Communcication, right-bottom; Incorrect Execution of Library Calls with Pending Communication.
28. Example program segments.
29. Power-cost relationship according to Grosch's law.
30. The Possible Speedup and Efficiency for Different $m$ and $n$.
31. Shared memory systems.
32. Shared memory via two ports.
33. Bus-based UMA (SMP) shared memory system.
34. NUMA shared memory system.
35. COMA shared memory system.
36. Write-Through vs. Write-Back.
37. Write-Update vs. Write-Invalidate.
38. Supervisor-workers model used in most parallel applications on shared memory systems.
39. Serial process vs. parallel process and two parallel processes communicate using the shared data segment.
40. Locks and barriers.
41. A sample OpenMP program along with its Pthreads translation that might be performed by an OpenMP compiler.
42. Message passing systems.
43. An example of a message passing system.
44. Hypercube broadcast tree-based communication.
45. Communication latencies in the store-and-forward (SF) and wormhole (WH) techniques.
46. Comparison Among a Number of Switching Techniques.
47. Typical SP 3 node.
48. Handshake for a blocking non-buffered send/receive operation.
49. Blocking buffered transfer protocols: (a) in the presence of communication hardware with buffers at send and receive ends; and (b) in the absence of communication hardware, sender interrupts receiver and deposits data in buffer at receiver end.
50. Space of possible protocols for send and receive operations.
51. Non-blocking non-buffered send and receive operations (a) in absence of communication hardware; (b) in presence of communication hardware.
52. Performance of Send/Receive on a Number of Message Passing Machines.
53. Using MPI_Comm_split to split a group of processes in a communicator into subgroups.
54. Representation of network technologies.
55. A multithreaded server in a client server system.
56. A socket connection.
57. Supervisor workers model in client server.
58. A cluster made of homogenous single-processor computers.
59. Source-path routing vs. table-based routing.
60. A 128-host Clos network using 16-port Myrinet switch (upper) and a 64-host Clos network using 16-port Myrinet switch (each line represents two links).
61. Quaternary fat tree of dimension 1 (left) and Elite switch of Quadrics networks (right).