the motivation for hadoop hadoop: basic concepts what is...

1compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

Section 9 : Case Study #Objectives of this Session The Motivation For Hadoop

What problems exist with traditional large-scale computing systemsWhat requirements an alternative approach should haveHow Hadoop addresses those requirements

Hadoop: Basic ConceptsWhat Is Hadoop?The Hadoop Distributed File System (HDFS)How Google MapReduce Algorithm worksAnatomy of a Hadoop Cluster

Who uses Hadoop ?

db.suven.net# Not a part of 1Z0-061 or 1Z0-144 Certification test , but very important technology in BIG DATA Analysis

• Hadoop Solutions

– The most common problems Hadoop can solve

– The types of analytics often performed with Hadoop

– Where the data comes from ?

– The benefits of analyzing data with Hadoop

– How some real-world companies use Hadoop

• Hadoop Ecosystem

• Cloudera Software (All Open-Source)

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177 2

Objectives of this Session … contd…

The Motivation For Hadoop

3compiled by Rocky Jagtiani Tech Head for

SCTPL , 9892544177

*MPI: Message Passing InterfacePVM: Parallel Virtual Machine


SCTPL , 9892544177

Major Problem


SCTPL , 9892544177

1 GB = 1000 MB , 1 TB = 1000 GB , 1 PT = 1000 TB , 1 Exabyte = 1000 PTPT => petabyte , TB => teraByte


SCTPL , 9892544177


SCTPL , 9892544177

The Motivation For Hadoop


SCTPL , 9892544177

1.

2.


SCTPL , 9892544177

3.

4.

5.

10

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

Hadoop History


SCTPL , 9892544177

Core Hadoop Concepts


SCTPL , 9892544177

Hadoop Components


SCTPL , 9892544177

HDFS


SCTPL , 9892544177

HDFSConcepts


SCTPL , 9892544177

HDFS : How Files Are Stored ?


SCTPL , 9892544177

How Files Are Stored: Example


SCTPL , 9892544177

IMP :

How MapReduce Work ?


SCTPL , 9892544177

MapReduce: The Mapper

19compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

Example :


SCTPL , 9892544177


25

Anatomy of a Hadoop Cluster :


Who uses Hadoop ?


29

Hadoop Solutions


A


31

B What is Problem if the data is coming ?


32

C


The most common problems Hadoop can solve :

We understand how each problem is solved using Hadoop in brief

D


35


40


41


42

How some real-world companies use HadoopE


43

Hadoop Ecosystem


Cloudera Software (All Open-Source)


45

*enterprise data

warehouse (EDW)

Conclusion :

1) Input to mapper is

"Google is one of the richest companies "

"one who works with the Google is technical expert "

what will be the out put after reducing ?


Questions


"Cat is eating milk"

"Cat is very sweet and she likes milk"

"milk is in bottle"




"Dollar is national currency for USA"

"Rupee is national currency for India"

"Dollar is ahead of Rupee in economy"

"India is developing country"

what will be the out put after Mapping ?




what will be the out put after shuffling?

the motivation for hadoop hadoop: basic concepts what is...

Documents