acadgild webinar - the correct time to switch to hadoop

presents

Webinar on

The Correct Time to Switch to Hadoop

Presented by: Shajee

© copyright ACADGILD

Brief Intro About AcadGild: CEO – Vinod Dham, Father of Pentium

2Big Data and Hadoop Development

• ACADGILD is a technology education start-up which provides online courses in

latest technologies like FrontEnd, FullStack, Big-Data, Android etc.

• Started by IIT/IIM alumni

• Our aim is to provide job ready skills to millions of high school and college

graduates, and working professionals.

Course Title© copyright ACADGILD

Is it the correct time to switch yourCareer with Hadoop?



Agenda Points


Sl No. Agenda Title

1 What is Big Data?

2 3 Vs of Big Data

3 From the Pen of Eric Schmidt- Ex-CEO, Google

4 Exploding Data Problem

5 Solution for Data Explosion – Hadoop

6 Core Components of Hadoop Cluster

7 Hadoop Ecosystem

Sl No. Agenda Title

8 Execution of First MapReduce Application

9 Job Prospects in Different Sectors

10 % Growth in Different Profiles

11 Companies Looking for Big Data Skills

12 Big Data-Related Job Titles

13 IDG Enterprise Big Data Research

14 Petrol Dataset Analysis using Pig


What is Big Data?



3 Vs of Big Data


Data Complexity

VolumeData Size

VelocitySpeed of Change

VarietyData Sources

• Terabytes• Records• Transactions• Table/Files

• Batch• Near-Time• Real-Time• Streams

• Structured• Unstructured• Semi-structured• All of the above


From the Pen of Eric Schmidt- Ex-CEO, Google


Every two days now we create as much information as we did from the dawn of civilization up until 2003, according to Schmidt. That’s something like five Exabyte of data.


Exploding Data Problem


• Big Data constitutes a large data set in PBs & ZBs which cannot be processed by a single machine within expected time frame.


Solution for Data Explosion - Hadoop


• Need a new System:• With new database management other than Relational Database, capable of

handling unstructured as well as structured data.• To process huge datasets on large clusters of computers, than on a single system.• To manage clusters in which: • Nodes fail frequently• Number of nodes keep changing• Common infrastructure which is:• Efficient• Easy to use• Reliable

Hadoop is that new system !!


Core Components of Hadoop Cluster


Hadoop 2.x Core Components

HDFS YARN

Storage Processing

NameNode

DataNode

Resource Manager

Node Manager

Master Layer

Slave Layer


Hadoop Ecosystem


Data Life Cylce &

GovernanceFalcon,Atlas

Data WorkFlow

SqoopFlumeKafka

Nfs

Provisi-oning

ManagingAmbari

OutBreakZooKeeper

SchedulingOozie

AdministrationAuthenticationAuthorization

AuditingData

ProtectionRangerKnoxAtlas

Governance Integration DATA ACCESS SECURITY OPERAT-

IONS

HDFS – Hadoop Distributed File System

YARN: Data Operating System

Batch

Map-Reduce

Script

Pig

NoSQL

HBase

Stream

Storm

SQL

Hive

Search

Solr

In-Mem

Spark

© copyright ACADGILD 12Big Data and Hadoop Development

Let’s execute our First MapReduce application


Job Prospects in Different Sectors


• According to Forbes, the top five industries who are hiring Big Data-related skills are Professional, Scientific and Technical Services, IT, Manufacturing, Finance, Insurance and Retail.

• The graph below shows the distribution of job openings in the above-mentioned sectors:


% Growth in Different Profiles


• Forbes also reported that the demand for sales representatives skilled in selling Big Data solutions are going through the roof and will continue to do so into 2016 as well as the upcoming years.

• Big Data-related jobs like Information Security Analysts, Management Analysts, Management Analysts and Information Security Analyst continue to be in high demand.


Companies Looking for Big Data Skills


Companies Looking for Big Data Skills:• EMC2, IBM, Cisco, Oracle are just a few of the top companies who are looking

for Big Data skills set. • Here’s a distribution of job requirements of the top ten Big Data employers

today, according to Wanted Analytics.


Big Data-Related Job Titles


• Here are some job titles that would provide you with full range of opportunities when looking for Big Data-related jobs.

• Take a look at it and expand your search:


IDG Enterprise Big Data Research


• According to IDG Enterprise Big Data Research, many organizations plan to invest in skill sets necessary for Big Data deployments, including Data Scientists, Data Architects, Data Analysts, Data Visualizers, Research Analysts, and Business Analysts in the next 12-18 months.


Petrol Dataset Analysis using Pig


Contact Info:

o Website : http://www.acadgild.com

o LinkedIn : https://www.linkedin.com/company/acadgild

o Facebook : https://www.facebook.com/acadgild

o Support: [email protected]


http://www.acadgild.com/

https://www.linkedin.com/company/acadgild

https://www.facebook.com/acadgild

mailto:[email protected]


Thank You

acadgild webinar - the correct time to switch to hadoop

Education