List of Figures

1. Abstraction Layers
2. View of the Field
3. SISD Architecture.
4. SIMD Architecture.
5. MIMD Architecture.
6. (a) MIMD Shared Memory, (b) MIMD Distributed Memory.
7. (a) SIMD Distributed Computers, (b) Clusters.
8. Two SIMD Schemes.
9. Two MIMD Categories; Shared Memory and Message Passing MIMD Architectures.
10. Cooperative-Communicating with other processes.
11. One sided-Communicating with other processes.
12. Example program segments.
13. Power-cost relationship according to Grosch's law.
14. MPI messages.
15. Data+Envelope.
16. MPI basic datatypes for C.
17. Handshake for a blocking non-buffered send/receive operation.
18. Blocking buffered transfer protocols: Left: in the presence of communication hardware with buffers at send and receive ends; and Right: in the absence of communication hardware, sender interrupts receiver and deposits data in buffer at receiver end.
19. Non-blocking non-buffered send and receive operations Left: in absence of communication hardware; Right: in presence of communication hardware.
20. Space of possible protocols for send and receive operations.
21. Diagram for Broadcast.
22. Diagram for Reduce.
23. Diagram for Gather.
24. Diagram for All_Gather.
25. Diagram for Scatter.
26. Diagram for Alltoall.
27. Using MPI_Comm_split to split a group of processes in a communicator into subgroups.
28. Left: Unix process. Right: Threads within a Unix process.
29. Threads model.
30. Thread shared memory model.
31. Threads Unsafe! Pointers having the same value point to the same data.
32. Passing single argument to thread function.
33. Passing multiple arguments to thread function via a structure.
34. Threads joining.
35. A representative sequence for using condition variables.
36. A sample OpenMP program along with its Pthreads translation that might be performed by an OpenMP compiler.
37. Representation of network technologies.
38. A multithreaded server in a client server system.
39. A socket connection.
40. Supervisor workers model in client server.
41. A cluster made of homogenous single-processor computers.
42. Source-path routing vs. table-based routing.
43. A 64-host Clos network using 16-port Myrinet switch (each line represents two links).
44. A 128-host Clos network using 16-port Myrinet switch, which includes 24 Xbar16s.
45. Quaternary fat tree of dimension 1 (left) and Elite switch of Quadrics networks (right).