big dat ppt

Presented By- SHAILJA DALMIA 13IT252

BIG DATA ANALYTICS USING HADOOP

INTRODUCTIONEra of digitilized WorldChallenges to cutting edge businessesGFS and MapReduceIn 2006,Mike Caferella & Doug Cutting

working under Nutch project implemented Hadoop.

Open Source Framework for writing and running distributed applications.

WHAT IS BIG DATA?

WHY DFS?

What is Distributed File System?

What is Hadoop?

Hadoop Core Components

What is HDFS?

Design of HDFS

Area where HDFS is not a good fit

HDFS COMPONENTS

NameNodeDataNodes

Job Tracker and Task Tracker

HDFS Architecture

Map Reduce• Framework that assigns task to each

datanodes. Map Step-master node takes the input ,partition

it up into smaller sub problem,leading to multi level tree structure.

Reduce Step-Combine the results and generate the output

Each mapping operation is independent of other,Key value pair is generated ,sorters and shufflers are applied .

Parallelism offer fault tolerance,if one nodes fails ,still the work can be rescheduled.

Similar to Divide and Conquer technique. Does task in parallel to accomplish work in less

time.

Hadoop Key Features:

AccessibleRobustnessSimpleScalableCost EffectiveFlexibleFault Tolerant

Differences Between Hadoop and RDBMS

Hadoop Designed to scale out

architecture.Key value pairsFunctional

Programming(scripts and codes),can build complex models

Offline processing (WORA)

RDBMSScaling is expensiveTables having relational

structureDeclarative queriesOnline Processing.(work

for random reading and writing few records.

Hadoop Related TechnologiesAvro-Data Serialization System,rich data

structures,container file,compact fast binary data format.

Chukwa-powerful toolkit for analyzing data.

Hbase-Distributed database,provides big table like capabilities.

Hive-data warehouse useful for data summarization .Uses HiveQL language.

ConclusionHadoop had gained huge momentum

Technologies around are evolving really fast

There is no “One size fits all”

Valuable ,powerful tool.

More targeted businesses.

THANK YOU