![Page 1: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/1.jpg)
3/12/2013 Computer Engg, IIT(BHU) 1
PARALLEL COMPUTERS-1
![Page 2: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/2.jpg)
INTRODUCTION
•A parallel computer is either a single computer with more than one processor or group of computers to run any application
![Page 3: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/3.jpg)
TYPES OF PARALLEL COMPUTERS•There are basically two types of parallel computers
-multiprocessor system
-multicomputer system
![Page 4: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/4.jpg)
MULTIPROCESSOR SYSTEM
•Through interconnection networks multiple processors are joined
•Each processor shares a global address space
•Each processor has a local memory
•For communication between Pi and Pj , Pi first writes in the shared address space then Pj reads it from the shared address space
![Page 5: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/5.jpg)
A MULTIPROCESSOR MODEL
Shared Address Space
INTERCONNECTION NETWORK
P1 P2 P3 Pn
![Page 6: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/6.jpg)
•A high level language with parallel programming constructs and compiler directive can be used for programming in such a multiprocessor system
•Threads(regular high level code sequences for individual processors) can also be used for the programming
![Page 7: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/7.jpg)
EXAMPLES OF MULTIPTOCESSORS•Dual Pentium
•Quad Pentium etc.
![Page 8: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/8.jpg)
ADVANTAGE
-Data is easily shared DISADVANTAGE
-Hardware implementation is difficult
![Page 9: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/9.jpg)
•Large shared memory multiprocessor system may have hierarchical and distributed memory
•In hierarchical memory arrangement accessing nearby memory locations is much faster, hence termed as NUMA “Non Uniform Memory Access”
•Similarly UMA provides “Unform Memory Access”
![Page 10: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/10.jpg)
MULTICOMPUTER SYSTEM
•Complete computers are connected through interconnection networks
•Each computer has a processor and local memory
•With the help of interconnection networks, computer communicate with each other
•Memory of a computer is not accessible by other processor
![Page 11: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/11.jpg)
•Computers communicate by passing messages
•A message is nothing but data or instruction to be sent
![Page 12: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/12.jpg)
Chip Multiprocessor
•There is considerable diversity among parallel machines
•One of the today’s popular parallel architectures is Multi Core architecture
•Multiple instruction execution engines are fabricated on a single chip
•These engines are known as Cores
![Page 13: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/13.jpg)
Features of Intel Core Duo
•Two 32-bit Pentium processors on a single chip
•Each processor has its own L1 data and instruction cache
•Shared L2 cache
•Shared memory controller, I/O controller
•Fast communication between two through shared memory
![Page 14: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/14.jpg)
LOGICAL ORGANIZATION OF INTEL CORE DUO
FRONT SIDEBUS
MEMORY BUS CONTROLLER
L2 CACHE
P1 P2
L1-I L1-D L1-I L1-D
![Page 15: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/15.jpg)
Other Parallel Computing Scenarios•Cluster computing
•Distributed computing
•Grid computing
•Cloud computing
![Page 16: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/16.jpg)
Cluster computing
Cluster: Multiple interconnected personal computers as parallel computing platform
- Interconnects are available in several forms including Gigabit ethrrnet, Myrinet, Infinibands, Fiber channels etc.
- Processor communicates with other processor by passing messsages
![Page 17: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/17.jpg)
A TYPICAL CLUSTER
Connection to other computers & switches in network
COMPUTERS
ETHERNET SWITCH
![Page 18: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/18.jpg)
Advantages
Advantages over most other forms of High Performance Computing:
•High scalability (Latest processors can easily be incorpoated)
•Fault tolerance
•Memory is not shared among the machines
![Page 19: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/19.jpg)
Distributed Computing
•Multiple autonomous computer work together to achieve a common goal
•Autonomous computers communicate through a computer network
•It runs as a single system
![Page 20: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/20.jpg)
A DISTRIBUTED SYSTEM
: NODE
![Page 21: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/21.jpg)
•In distributed computing each processor has its own private memory compared to parallel computers which has shared memory.
Shared Memory
P1 P2 Pn
![Page 22: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/22.jpg)
Benefits Over Centralized System•Scalability: System is easily expanded by adding more machines
•Redundancy: Several machines solve same problem
![Page 23: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/23.jpg)
Grid Computing
•Notion of computing grid arose in the early 1990’s
•Like Power Grid where anyone can tap into, without worrying how the power is obtained and transmitted, A computing grid provides computational power hiding other details from the user
•Resources located at different place participate in a grid
•Grid computing harnesses the CPU cycles of the computers in a network to solve a problem
![Page 24: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/24.jpg)
Primary Attributes of a Grid
Ian foster listed 3 primary attributes of a grid:
•Computing resources are not administered centrally
•Open standards are used
•Non-trivial quality of service is achieved
![Page 25: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/25.jpg)
GARUDA Grid
•India’s first national grid initiative
•Academicians, researchers together develop their data and compute intensive applications
•GARUDA is a SOA based cyber infrastructure connecting computational nodes, mass storage and scientific instruments
![Page 26: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/26.jpg)
Cloud Computing
•Cloud uses internet and central remote servers to maintain data and applications
•Provides computation, software, data access and storage resources
•A cloud can be used through a web browser or simple desktop/mobile apps.
![Page 27: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/27.jpg)
Cloud Computing Service Models
Application
Infrastructure
Platform
Server
Tablet
Laptop
Mobile
Desktop
![Page 28: 3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1](https://reader036.vdocuments.site/reader036/viewer/2022081515/5a4d1b507f8b9ab0599a7354/html5/thumbnails/28.jpg)
REFERENCES
•Wilkinson