MPI Raspberry pi 3 & Hadoop cluster

4 nodes

MPI Raspberry pi cluster

Why MPI Hardware Needed Software Needed Machine File MPI Code Python Code Output Finished product

Why MPI?

Scientist during World War II used the same method of Parallel Computing to solve mathematical problems in the Manhattan Project. reduce the amount of time it took to solve a large mathematical problem. the people who performed these calculations were called computers.

How far can you go with MPI?

Hardware you need

X4 Raspberry pi 3X5 Ethernet cat 5 cablesX4 micro USB cablesX4 8 GB micro SDOne 5 port Ethernet switchOption but recommended one 4 port USB power hub

Software Needed

MPICH

MPI4PY Python to MPI interpreter

Machine File

MPI code

Python code

Output

Finished product

Hadoop

Hadoop is a framework of tools based on Java that supports running Big Data.

Inspired from Google’s GFS and MapReduce. It is open source. Relatively cheap to build. Has Fault Tolerant System.

Parts of Hadoop

Hadoop consists of two main elements, HDFS (Hadoop Distributed File System)

Name Node (Store the directories of all files in the cluster) Data Node (Store Data)

MapReduce Job Tracker(Responsible for taking requests from client and assign to

TaskTracker) Task Tracker(Perform MapReduce in DataNodes/Slave)

HDFS (Hadoop Distributed File System)

MapReduce

Use of Hadoop in Tech World

Amazon Facebook Netflix eBay Twitter etc.

Thank You