LECTURE #1

INTRODUCTON TO PARALLEL COMPUTING

1. What is parallel computing?2. Why we need parallel computing?3. Why parallel computing is more difficult?4. What are uses for Parallel Computing in Science and

Engineering?5. What are uses for Parallel Computing in Industrial

and Commercial?6. How can we do Parallel computing in Laptop or

Desktop?7. Briefly write the Parallel Computer Examples?

LECTURE #2

PARALLEL COMPUTER ARCHITECTURES

Flynn's Classical Taxonomy

The 4 possible classifications according to Flynn: Single Instruction, Single Data (SISD) Single Instruction, Multiple Data (SIMD) Multiple Instruction, Single Data (MISD) Multiple Instruction, Multiple Data (MIMD)

1. Single Instruction, Single Data (SISD)

2. Single Instruction, Multiple Data (SIMD)

3. Multiple Instruction, Single Data (MISD)

4. Multiple Instruction, Multiple Data (MIMD)

Classification for MIMD Computers

MIMD Computers are classified into two types:

Shared Memory

Distributed Memory

Shared Memory

Distributed Memory

Assignment #1

Briefly Explain Systolic Architecture.(For Even Roll No.)

Briefly Explain Vector Architecture.(For Odd Roll No.)

Important Note:1. Not more than 1 and Half paper.2. Write your Name, Roll No., Section and Batch on top of paper.3. Assignment must be submitted in next week.4. Assignment must be Hand Written.

LECTURE #3

CONCEPTS AND TERMINOLOGY OF PARALLEL COMPUTING

CONCEPTS AND TERMINOLOGY OF PARALLEL COMPUTING

1. High Performance Computing (HPC)2. Node3. Task4. Pipelining5. Symmetric Multi-Processor (SMP)6. Synchronization7. Granularity8. Multiprogramming9. Multiprocessing10. Multitasking11. Simultaneous Multithreading (SMT)

LECTURE #4

PARALLEL COMPUTING MATRIC

Amdahl’s LawAmdahl’s Law calculates the speedup of parallel code based on three variables:Duration of running the application on a single-core machine.The percentage of the application that is parallel.The number of processor cores.

Here is the formula, which returns the ratio of single-core versus Multicore performance.

The variable P is the percent of the application that runs in parallel, and N is the number of processor cores.

Numerical # 1Suppose you have an application that is 75 percent parallel and runs on a machine with three processor cores. Find Speedup by using Amdahl’s Law?

Numerical # 21. If you have an algorithm in which only 50 percent (P =

0.50) of its total work is executed in parallel with two physical cores. An algorithm with 1,000 units of work split into 500 units of sequential work and 500 units of parallelized work. Find Speedup by using Amdahl’s Law?

2. If the sequential version of code takes 1,000 seconds to complete then how long, the new version with some parallelized code will take?

Numerical # 31. The maximum speedup for the same algorithm on a

microprocessor with eight physical cores. Find Speedup by using Amdahl’s Law?

2. If the sequential version of code takes 1,000 seconds to complete then how long, the new version with some parallelized code will take?

Numerical # 4Find the maximum speedup for the algorithm according to the number of physical cores, from 1 to 16. In algorithm in which 90 percent (P = 0.90) of its total work is executed in parallel?

Gustafson’s LawGustafson’s Law provides the following formula with the focus on the problem size to measure the amount of work that can be performed in a fixed time:

Total work (in units) = S + (N × P)

where:S represents the units of work that run with a sequential execution.P is the size of each unit of work that runs completely in parallel.N is the number of available execution units (processors or physical cores).

Numerical # 5 You can consider a problem composed of 50 units of work with a sequential execution. The problem can also schedule parallel work in 50 units of work for each available core. If you have a microprocessor with two physical cores, find the maximum amount of work ?

Numerical # 6 The same algorithm can run on a microprocessor with eight physical cores. Find the maximum amount of work ?

Assignment # 2Suppose you have an application that is 45 percent parallel and runs on a machine with three processor cores. Find the Speed Up by using Amdahl’s Law. (For Even Roll No.)

If you have an algorithm in which only 30 percent (P = 0.30) of its total work is executed in parallel, a microprocessor with two physical cores. Find the Speed Up by using Amdahl’s Law?(For Odd Roll No.)

Important Note:1. Not more than 1 and Half paper.2. Write your Name, Roll No., Section and Batch on top of paper.3. Assignment must be submitted in next week.4. Assignment must be Hand Written.

LECTURE # 5

LEVEL OF PARALLELISM

Levels Of Parallelism

1. Instruction Level Parallelism 2. Thread Level Parallelism3. Process Level Parallelism4. Message-Passing Parallelism

Assignment # 3

Why fine grained Parallel Processing is potentially faster ?

Important Note:1. Not more than 1 paper.2. Write your Name, Roll No., Section and Batch on top of paper.3. Assignment must be submitted in next week.4. Assignment must be Hand Written.

LECTURE # 6

INTERCONNECTION NETWORKS - I

• What is Interconnection Networks?• Types of Interconnection Networks

Interconnection Networks:

Shared versus Switched Media

BUS:

CROSSBAR NETWORK:

MULTISTAGE NETWORKS:

LECTURE # 7

INTERCONNECTION NETWORKS - II

HYPERCUBE NETWORKS:

Mesh Networks:

TREE NETWORKS:

BUTTERFLY NETWORKS::

PYRAMID NETWORKS:

STAR NETWORK

QUIZ # 2Q1) What is Interconnection Networks?

Q2) Briefly defined Cross Bar Network?

Q3) Briefly defined Butterfly Network?

Q4) Briefly defined Hypercube Network?

Important Note:1. Write your Name, Roll No., Section and Batch on top of paper.2. Time :15 Minutes .