hybrid cloud architectures for operational - elysium · hybrid cloud architectures for operational...

BOEING is a trademark of Boeing Management Company

Copyright © 2014 Boeing. All rights reserved.

Copyright © 2014 Northrop Grumman Corporation. All rights reserved.

GPDIS_2014.ppt | 1

Hybrid Cloud

Architectures

for

Operational

Performance

Management

Delbert Murphy

Solution Architect / Data Scientist

Microsoft Corporation

Global Product Data Interoperability Summit | 2014




GPDIS_2014.ppt | 2

Delbert Murphy and Microsoft’s Data Insights CoE

Microsoft Azure

SQL Server

Power BI

CustomerAnalytics

Technologies Areas of Expertise

HDI Insight

Azure Machine Learning

OperationalIntelligence

Analytics for Finance





GPDIS_2014.ppt | 3

Quick census

Information TechnologyEngineering

?

Other

ManufacturingDesign ManagerDBAArchitect DeveloperData Scientist





GPDIS_2014.ppt | 4

The NIST cloud definition framework

Hybrid CloudsDeployment

Models

Service

Models

Essential

Characteristics

Common

Characteristics

SaaS

PaaS

IaaS

Resource Pooling

Broad Network Access Rapid Elasticity

Measured Service

On-Demand Self-Service

Low-Cost Software

Virtualization Service Orientation

Advanced Security

Homogeneity

Massive Scale Resilient Computing

Geographic Distribution

Hosted

Cloud

Public

Cloud

Private

Cloud





GPDIS_2014.ppt | 5

Event volume





GPDIS_2014.ppt | 6

Event velocity





GPDIS_2014.ppt | 7

Event source variety

1B

Connected/Smart TVs

2.5B

Personal computers

>5B

Smartphones and tablets

>10B

Connected internet of things

Source: Gartner, IDC, Strategy Analytics, Machina Research, Company filings, BI Intelligence, Accenture

analysis

Copyright © 2013 Microsoft and Accenture

Revenue engines

Content Applications Apps and services Services by verticals





GPDIS_2014.ppt | 8

Event veracity

ve·rac·i·ty

1. Devotion to the truth: truthfulness

2. Power of conveying or perceiving the truth

3. Conformity with truth or fact: accuracy

Record the confidence a source system has in the event value

Use a large enough N to confident in the value





GPDIS_2014.ppt | 9

Grid

Renewables

Oil/Gas/Coal

Recovery and

Distribution

Points

of SaleRestaurants

Hotels

Fuel

Stations

Patients

Clinics

Hospitals

Nursing

Homes

Mobile

Care

SafetySecurity

ComfortLighting

Automation

Manufacturing

Integration and

Automation

Remote

Servicing

Predictive and

Reactive

Maintenance

Water

Waste

Pollution

Control

Fire

Emergency

Public

Safety

Law

EnforcementLetters

Packages

Containers

Tanks Bulkware

Games

Events

Sports

TelevisionStreaming

Traffic Buses

Cars

Trucks

Trains

Vessels

Aircraft

Bikes

Smart

Retail

Smart

Mobility

Smart

Logistics

Smart

Factory

Smart

Entertain

-ment

Smart

Health-

care

Smart

Building

/ Home

Smart

Energy

Smart

Cities

Event impact





GPDIS_2014.ppt | 10

Scenarios for operational performance management

Predictive Maintenance

• Avoid costly asset downtime and reduce maintenance costs

• Reduce warranty claims due to unexpected failures

• Improve inventory management of spare parts by predicting failures

Performance Management

• Improve Production Quality Assurance and Yield

Event and Incident Management and Monitoring

• Better protect Health, Safety and Environment

• Improve Situational Awareness, Security and Tracking

Route and Capacity optimization

• Optimize Power grids and networks

• Optimize Traffic and Goods movement





GPDIS_2014.ppt | 11

Process

Insights

•Business Intelligence

•Train predictive analytics models

Consume

•Exception alerting

•Query predictive analytics models

Ingest

•Fast and scalable

Transport

•Communication

•Store-and-Forward?

Emit

•Exhaust data

Design

•Business Objectives

•KPI’s

• Information requirements

Visual Studio Visual Studio

(device dependent)

ISS

Event hug

ISS

Reykjavik

Azure NRT

ISS

Reykjavik

Azure NRT

Azure ML

PowerBI

HDInsight

Azure ML





GPDIS_2014.ppt | 12





GPDIS_2014.ppt | 13

Device communication patterns





GPDIS_2014.ppt | 14

Device communication styles





GPDIS_2014.ppt | 15

Device communication scope





GPDIS_2014.ppt | 16

Selecting the right components

Buy

Build

Buy Some, Build Some

Focus on Business Result and Time To Value

Mix and Match

• Custom protocols & security

models

• Custom analytics and data

processing

Core Competency in building software &

Services

• Focus on flexibility & control

• Strategic investment in service platform





GPDIS_2014.ppt | 17

Azure ISS, no custom code

• Limited experience in software and service development

• Very sensitive to time to market and budget

• Connect remote kiosks to ISS via Windows agents for basic device management

Scenarios and Workloads:

• Entire System Management

• Station Management

• Mobile Maintenance Worker Tasking and Support

Azure Platform Service

ISS Component

Reference Component

Custom Code

ISS Agent

ISS Service

Device RegistryDevice Mgmt Services

Service Bus (Event Hub)

Event Processing

Azure Storage

Data Services

Kiosk ... Kiosk

Device

ISS Device Operator Portal

ISS Agent ISS Agent





GPDIS_2014.ppt | 18

Custom devices & portal on Azure ISS

• Depth and experience in software and service development

• Very sensitive to time to market (more buy than build)

• Connect sensors to ISS agents, push data to ISS

• Command and control from ISS

• Other Azure services used - AAD

Scenarios and Workloads:

• Entire System Management

• Station Management

• Mobile Maintenance Worker Tasking and Support


ISS Component

Reference Component

Custom Code

ISS Agent

ISS Service



Event Processing

Azure Storage

Data Services

Device Mgmt API Data API (OData)Custom Portal

AzureActiveDirectory

Windows Gateway

PLC

Escalator Elevator Powerlines

Device

Maintenance UX on Mobile Device





GPDIS_2014.ppt | 19

Custom protocol, to cloud gateway to ISS

• Limited depth and

experience in software

development

• Good fit for turn-key service

platform

• Front end connectivity and

data flow with Reykjavik

• Flow data into ISS for

processing, device

management

• ISS publishes data to

customer storage account

for batch processing

• ISS publishes copy of live

data stream to customer

Event Hub for custom

processing

Custom Device (no agent – RTOS, etc)


ISS Component

Reference Component

Custom Code

ISS Agent

Service Assisted Connectivity

ISS Service



Event Processing

Azure Storage

Data Services

Device Mgmt API Data API (OData)

Custom Protocol Gateway (Reykvavik)

Protocols

Schemas

Security


Message Pump

Storm

HDInsight

SQL DB

Protocols

Schemas

Security

Azure Storage





GPDIS_2014.ppt | 20

Custom protocols and Azure building blocks

• Depth and experience in software and service development

• High demand for control and flexibility (more build than buy)

• Needs custom protocol support for extant devices

• Build out on Azure services using Reykvavikcomponents to streamline delivery

Custom Device (no agent – RTOS, etc)


ISS Component

Reference Component

Custom Code

ISS Agent

Service Assisted Connectivity

WorkerRole

Custom Code

ISS Service



Event Processing

Azure Storage

Data Services

Device Mgmt API Data API (OData)

Custom Protocol Gateway (Reykvavik)

Protocols

Schemas

Security


NRT

Message Pump

Azure Storage HDInsight

SQL DBAMQP

AMQP

Protocols

Schemas

Security





GPDIS_2014.ppt | 21

The right strategy

Generate new

insights to

create new

business value

Combine the

data you

already collect

Expand by adding new devices, new services, new data

Utilize services

and the cloud

to jump-start

your efforts

Start by

connecting the

devices you

already have





GPDIS_2014.ppt | 22

Intelligent (cognitive) systems

Text

HereSuper

ProcessingIntelligent AgentsEnvironment

actions

percepts

data

algorithmsRussell and Norvig:

AI An Agent-Based Approach





GPDIS_2014.ppt | 23

Lidberg’s law

Data born in the cloud, stays in the cloud. Data born on

premised stays on premises.

Simon Lidberg, Microsoft





GPDIS_2014.ppt | 24

Homan’s law

Think carefully about where you place your data: it

forms a gravity well. A data storage mechanism will

attract other data in direct proportion to the amount of

data it currently holds.

Uli Homan, Microsoft





GPDIS_2014.ppt | 25

Murphy’s law of distributed data

Don’t distribute your data.

Compute locally, to join globally.

Delbert Murphy, Microsoft





GPDIS_2014.ppt | 26

Information hierarchy

Data

Information

Knowledge

Understanding

Wisdom

Individual Group





GPDIS_2014.ppt | 27

Heuristics

Heuristics AnalyticalSynthesis





GPDIS_2014.ppt | 28

Questions and answers





GPDIS_2014.ppt | 29

Appendix





GPDIS_2014.ppt | 30

Microsoft’s data platform stack





GPDIS_2014.ppt | 31

Microsoft’s data warehouse stack





GPDIS_2014.ppt | 32

Microsoft’s modern data platform





GPDIS_2014.ppt | 33

Microsoft data platform and types of data

hybrid cloud architectures for operational - elysium · hybrid cloud architectures for operational...

Documents