what an it leader needs to know about bigdata (it leaders event september 2015)

03/05/2023 1

WHAT THE IT LEADER NEEDS TO KNOW ABOUT BIG DATA

John MortonExecutive Leaders NetworkSeptember 2015

Executive Leaders Network – September 2015

http://creativecommons.org/licenses/by-nc/4.0/

Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.

Executive Leaders Network – September 201503/05/2023 2

In 2011 (Gartner) this now includes – Complexity!! May 2011 McKinseys Report “Big Data : The next frontier for Innovation,

Competition and Productivity” stated : US healthcare reduction by $300 B a year 2/3rds from a 8% reduction

in national healthcare Retailers can increase operating margin by 60% by fully utilising data Euro 100 Bn reduction in Government administration across Europe

Big data is a term applied to data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. Big data sizes are a constantly moving target currently ranging from a few dozen terabytes to many petabytes of data in a single data set. (Wikipedia) In a 2001 (Gartner), defined data growth challenges (and opportunities) as being three-dimensional, increasing volume (amount of data), velocity (speed of data in/out), and variety (range of data types, sources).

WHAT IS BIG DATA?

http://en.wikipedia.org/wiki/Gartner




Predict ActAct

Appliances

Structured

SAP

Internet

Windows

Semi-Structured

Other Sources

Un-Structured

Real Time

High Volume Batch

Web Services

Message Queues

Sense




USING ANALYTICS

Com

petit

ive

Adva

ntag

e

Intelligent insight

Raw Data

Business Maturity

Cleaned Data

Standard Reports

Cubes and dashboards

Alerts

Explore

Predict

Optimise

What has happened?

What is the frequency?

Where is the problem?

What actions are needed?

What is the best outcome?

What may happen next?

Why is it happening?




WHAT DOES DO?

Analytics

Data Intelligence

Customer Intelligence

Business Intelligence

Fraud and Risk




EIM FRAMEWORKVision

& Strate

gy

Governance

EIM Core Processes Organization

Tools & Technologi

es

Vision

Mission

StrategyGoals

& Objectives

Sponsorship

Stewardship

Policies, Principle

s &Tenets

Alignment

Structure

Data QualityData IntegrityData Security & Protection

Data LifecycleData

MovementSemantics Management

Database ManagementMaster Data ManagementInformation

Services (A&R)Services &

Support

Value Proposi

tions

CSFs & KPIs

Structure (Virtual,

Hybrid, …)Roles &

Responsibilities

Functional ServicesBusiness Value &

Relationship Managemen

t

Information

Architecture

Blueprint Manageme

nt

Technologies (DBMS, CMS, ETL, EAI, EII,

Data Modeling,

BI/DW, Collaborati

on)Knowledge

base & Repositori

esStandards

& Best Practices




DATA LAKE

IT Business




DATA LAKE FRAMEWORKVision

& Strate

gy

Governance

EIM Core Processes Organization

Tools & Technologi

es

Vision

Mission

StrategyGoals

& Objectives

Sponsorship

Stewardship

Policies, Principle

s &Tenets

Alignment

Structure

Data QualityData IntegrityData Security & Protection

Data LifecycleData

MovementSemantics Management

Database ManagementMaster Data ManagementInformation

Services (A&R)Services &

Support

Value Proposi

tions

CSFs & KPIs

Structure (Virtual,

Hybrid, …)Roles &

Responsibilities

Functional ServicesBusiness Value &

Relationship Managemen

t

Information

Architecture

Blueprint Manageme

nt

Technologies (DBMS, CMS, ETL, EAI, EII,

Data Modeling,

BI/DW, Collaborati

on)Knowledge

base & Repositori

esStandards

& Best Practices




DATA LAKE FRAMEWORK CAPABILITIES

INFORMATION GOVERNANCESupporting

ServicesEvents, linkage

Workflow & Business

Rules

Information Quality

and Information metrics

MDM

INFORMATION AND CONTENT ACCESS

INFRASTRUCTURE SERVICESSecurity, Meta-data & Lineage, Versioning, Monitoring &

Deployment

Algorithm Managemen

t, Calibration

& Monitoring

Information and

Algorithm Assessment

INSIGHT SERVICES

Interfaces and

Integration




VALUE OF DATA LAKESC

umul

ativ

ebu

sine

ss v

alue

time

uncertainty decreases over time

unce

rtain

ty




TWO WORLDSInformation

Prep

Information Analyse

Develop Model

Calibrate Model

Information PreparationInformation ExplorationInformation VisualizationReport Creation

Exploratory AnalysisWhat if assessmentCorrelationsDescriptive SegmentationPredictive Modeling

DataAnalyst

AnalyticsAnalyst

Industrialise Model

Validate Model

Deploy Model

Evaluate and Calibrate

Model ValidationModel DeploymentModel Monitoring Information Preparation

IT Operations

Evaluates processesAssesses EffectivenessCalibrates model

Business Operations

Frames Problem in contextEstablishes metrics of successAgrees outcomes

Business Analyst




CEO

Function

Function

Function

Function

CEO

Function

Function

Function

Function

CEO

Function

Function

Function

Function

Function

Leading analytics CMO + CIO + CPO + COO £

ORGANISATION STRUCTURES




BUSINESS ANALYTICS ECO-SYSTEM

Data assets

Analyticsstrategy

Value propositions

ICT strategy

HR strategy

Business strategy

Vidgen, R., (2014). Creating business value from Big Data and business analytics: organisational, managerial and human resource implications. Hull University Business School Research Memorandum, no. 94, ISBN 978-1-906422-31-8.




WHAT ARE YOU MISSING?




PARTING THOUGHTS1. Can I trust that the technology chosen is

going to get the best out of my data?2. What are the trade-offs between the

algorithms employed by “service A” versus “service B”?

3. If the solution claims to be automated analytics, how robust is that automation?

4. Is the technology “extensible” and “interoperable” to meet changing demands from data, evolving business, or competition?




THOUGHTS ?

Eur. Ing. John Morton BSc, CEng. FBCS, CITP, [email protected] +44 7771 740203Thanks to Pixabay for Images


what an it leader needs to know about bigdata (it leaders event september 2015)

Technology