Contents

• List of Tables
• List of Figures
• Introduction
  • Four Decades of Computing
  • Flynn's Taxonomy of Computer Architecture
  • SIMD Architecture
  • MIMD Architecture
    • Shared Memory Organization
    • Message Passing Organization
  • Interconnection Networks
  • Introduction to MPI
    • Parallel Computing
    • Types of parallel computing
    • Communicating with other processes
    • Hardware models
    • What is MPI?
    • Who Designed MPI?
    • MPI Implementations
    • Is MPI Large or Small?
    • Where to use MPI?
    • How To Use MPI?
    • Getting started
  
  
• Multiprocessors Interconnection Networks
  • Interconnection Networks Taxonomy
  • Bus-based Dynamic Interconnection Networks
    • Single Bus Systems
    • Multiple Bus Systems
    • Bus Synchronization
  • Switch-Based Interconnection Networks
    • Crossbar Networks
    • Single-Stage Networks
    • Multistage Networks
    • Blockage in Multistage Interconnection Networks
  • Static Interconnection Networks
    • Completely Connected Networks
    • Limited Connection Networks
  • Analysis and Performance Metrics
    • Dynamic Networks
    • Static Networks
  • Sending and Receiving messages
    • Current Message-Passing
    • The Buffer
    • Sample Program using Library Calls
    • MPI Basic Send/Receive
    • Exercise/Example: Translate this Fortran http://siber.cankaya.edu.tr/ParallelComputing/cfiles/sendreceive.fcode to C
    • Computation of PI as an example
    • Exercise
  • MPI Hands-On; Examples
  
  
• Performance Analysis
  • Computational Models
    • Equal Duration Model
    • Parallel Computation with Serial Sections Model
  • Skeptic Postulates For Parallel Architectures
    • Grosch's Law
    • Amdahl's Law
    • Gustafson-Barsis's Law
  • Scalability of Parallel Architectures
  • Assignment:
  • Performance tuning MPI 1
  
  
• Shared Memory Architecture
  • Classification of Shared Memory Systems
    • Uniform Memory Access (UMA)
    • Nonuniform Memory Access (NUMA)
    • Cache-Only Memory Architecture (COMA)
  • Bus-based Symmetric Multiprocessors
  • Basic Cache Coherency Methods
    • Cache-Memory Coherence
    • Cache-Cache Coherence
    • Shared Memory System Coherence
  • Shared Memory Programming
    • Task Creation
    • Communication
    • Synchronization
    • Thread Basics
  • Performance tuning MPI 2
  
  
• Programming Shared Memory
  • Why Threads?
  • Thread Basics: Creation and Termination
  • Synchronization Primitives in Pthreads
    • Mutual Exclusion for Shared Variables
  • Shared Memory I; Processes, Threads
    • Condition Variables for Synchronization
  • Controlling Thread and Synchronization Attributes
    • Attributes Objects for Threads
  • Thread Cancellation
  • Composite Synchronization Constructs
    • Barriers
  • Tips for Designing Asynchronous Programs
  • OpenMP: a Standard for Directive Based Parallel Programming
    • The OpenMP Programming Model
  • Assignment:
  • Shared Memory II; Threads, OpenMP Examples
  
  
• Message Passing Architecture
  • Introduction to Message Passing
  • Routing in Message Passing Networks
    • Routing for Broadcasting and Multicasting
    • Routing Potential Problems
  • Switching Mechanisms in Message Passing
  • Processor Support for Message Passing
  • Example Message Passing Architectures
  • Message Passing vs Shared Memory
  • Parallel Summation
  
  
• Programming Using the Message-Passing Paradigm
  • Principles of Message-Passing Programming
    • Structure of Message-Passing Programs
  • The Building Blocks: Send and Receive Operations
    • Blocking Message Passing Operations
    • Non-Blocking Message Passing Operations
  • MPI: the Message Passing Interface
    • Starting and Terminating the MPI Library
    • Communicators
    • Getting Information
    • Sending and Receiving Messages
  • Examples; Buffer Limit, Barrier, Collective Communications.
  • Overlapping Communication with Computation
    • Non-Blocking Communication Operations
  • Collective Communication and Computation Operations
    • Reduction
    • Gather
    • Scatter
    • All-to-All
  • Groups and Communicators
  • Examples; Non-blocking Communications and Parallel Matrix Multiplication.
  
  
• Introduction to Parallel Algorithms
  • Odd-Even Sort
  • Cannon's Matrix-Matrix Multiplication with MPI's Topologies
  • One-Dimensional Matrix-Vector Multiplication
  • Two-Dimensional Matrix-Vector Multiplication
  • Examples; Parallel Algorithms
  • Network Computing
  • Computer Networks Basics
    • Network Performance
    • Internet
    • Other Network Technologies
  • Client/Server Systems
    • Sockets
    • Remote Procedure Call (RPC)
    • Middleware
    • A Client Server Framework for Parallel Applications
  • Clusters
    • Threads
  • Interconnection Networks
    • Ethernet
    • Switches
  • Myrinet Clos Network
    • The Quadrics Network
  • Cluster Examples
    • Berkeley Network of Workstations (NOW)
    • The Beowulf Cluster
    • FlashMob I
  • Grid Computing
  
  
• References:
• vita