project review1

7/31/2019 Project Review1

1/29

FILE FORMAT CONVERTER: AVI TO 3GP USING

HADOOP MAP REDUCE FRAMEWORK


2/29

TABLE OF CONTENTS

NEED

LITERATURE SURVEY HDFSHADOOP MAP REDUCEFORMAT CONVERSION

2


3/29


4/29

NEEDTHE MANY TO MANY MAPING:

4


5/29

NEED:

File playable in one device cannot be played in other

device. It has to be converted to other formats to be played. Hence we go for format conversion.

5


6/29


7/29

LITERATURE SURVEY

Existing tools:

Any to any convertor Format Factory Total video Convertor

7


8/29


9/29

LITERATURE SURVEY

To convert larger files at a faster rate,we go for cluster

environment.

Existing System in cluster Environment:

MAV GRID

9


10/29

LITERATURE SURVEY

This mav grid focuses on resource sharing but not onparallelizing the converting operation

So we go for HADOOP DISTRIBUTED FILE SYSTEM.

10


11/29


12/29

HDFS

Master/slave architectureHDFS cluster consists of a single Namenode , a masterserver that manages the file system namespace andregulates access to files by clients.There are a number of DataNodes usually one per node ina cluster.

12


13/29

13

HDFS Architecture

13

Namenode

Breplication

Rack1 Rack2

Client

Blocks

DatanodesDatanodes

Client

Write

ReadBlock ops


14/29

HDFS-DATA NODE

The DataNodes manage storage attached to the

nodes that they run on. A file is split into one or more blocks and set of blocks are stored in DataNodes.DataNodes serves read, write requests, performsblock creation, deletion, and replication uponinstruction from Namenode.

14


15/29


16/29

HDFS

The Namenode receives a Heartbeat and a

BlockReport from each DataNode in the cluster.BlockReport contains all the blocks on a Datanode.Receipt of a Heartbeat implies that the DataNodeis functioning properly.

16


17/29


18/29

HADOOP MAP REDUCE Users Hadoop map reduce:

18


19/29

HADOOP MAP REDUCE With this project CIT joins the Line:

19


20/29

HADOOP MAP REDUCE

Moving the computation is cheaper than moving data, soinstead of moving the entire data for conversion, theproposed technique move the process of conversiontowards the data.

Exploits large set of commodity computers Executes process in distributed manner Processing large data set.

20


21/29

HADOOP MAP REDUCE

21


22/29

HADOOP MAP REDUCE

NAME NODE-------------- MAP -------------- JOB TRACKER

DATA NODE-------------REDUCE---------- TASK TRACKER

22


23/29

HADOOP MAP REDUCE

23


24/29


25/29


26/29

HADOOP MAP REDUCE

26

AVI FILEHDFS

MAP-CONVE

RT

REDUCE

-

COMBIN

E3GP FILE


27/29

REFERENCES

[1] Hadoop 0.20 Documentation[2] http://hadoop.apache.org/mapreduce/ [3]IEEE paper on A Design of Grid Supported Services forMobile Learning System by M. Norazizi Sham MohdSayuti,Universiti Sains Islam Malaysia (USIM)

27
http://hadoop.apache.org/mapreduce/http://hadoop.apache.org/mapreduce/


28/29


29/29

project review1

Documents