Copyri

ght

Life of an Event

A never ending tool chain

26-03-2015

Arnold van Wijnbergen

Copyri

ght

2

Arnold van Wijnbergen@BSMConsultant / Arnold.van.Wijnbergen@gmail.com

•15 years working experience

with automating IT

•DevOps Evangelist with

great passion about

Architecture and

Automation Tooling

•Full time Tooling Geek

Automation

Monitoring

Event management

Deployment

#IlovIT ☁

Copyri

ght

3

AND WHO ARE YOU ?

Dev, Ops or just DevOps

Copyri

ght

4

Do you recognize this situation?

Organization is having monitoring but

always missing the correct data and

triggers.

Outage top 10 week 12

Copyri

ght

5

REALITY CHECK

IT is missing some crucial

parts for monitoring

Copyri

ght

6

Most monitoring is reactive and based

on what we know

Monitoring must be proactive and

based on what we see as unusual

behavior

Copyri

ght

7

Here the story begins …

This requires a different approach,

which fits the DevOps culture

Copyri

ght

8

Next statements will place the

context to DevOps

By applying the three Ways

/Underpinning Principles of DevOps

(Gene Kim)

• Systems Thinking

• Amplify Feedback Loops

• Culture of Continual

Experimentation And Learning

Copyri

ght

9

ITOPS IS MISSING CONTROL

BUT

THIS IS A DEV&OPS CHALLENGE

Statement 1

Copyri

ght

10

Statement 2

DEV BUILDS BUSINESS VALUE

OPS MUST SUPPORT BUSINESS VALUE

Development started with Agile

Scrum approach.

DevOps will fill in these gaps for

Operations with use of Continuous

Monitoring to support the

Continuous Delivery pipeline

Copyri

ght

11

BUT

Statement 3

DEV LIKE MANY EVENTS

OPS LIKE FEW EVENTS

We need best of both worlds

Copyri

ght

12

BUT

Statement 4

DEV DOES TESTING

OPS DOES MONITORING

Align both disciplines

Close your feedback loop here

Reuse code for Quality

Assurance and Continuous

Improvement…

Copyri

ght

13

BUT

Statement 5

All resources within your solution

are involved. Operations must be

able to pinpoint the outage,

understand and improve the whole

solution to increase resilience

Monitoring is NOT only about

infrastructure and solving the problem

Copyri

ght

14

Now that we know the

DevOps part of the story

How should we fill in the

Monitoring part of the story ?

Copyri

ght

15

Application

Infra

structure

Business

Activity

Transaction

Data

Application

Runtime

Middleware

OS

Virtualization

Server

Storage

Network

Automation glue

Monitoring all Resources …

• Their characteristics and

variables need to be measured

as an indicator to build up an

expected health state.

• Send out events, which give you

specific information about a

resource, such as availability

and software faults.

• Generate metric data, which

give you time-series analyse on

a resource their performance.

But individual resources don’t

give you a reliable overview of

your solution/enterprise state

!!!

Customer SolutionFirst the WHAT ?

Copyri

ght

16

Just like a starry sky all events (like stars) are inter-related

to each other like (the constellation)

And sometimes a star falls down, which breaks the chain

Copyri

ght

17

Now that we talked you through Space

we will take the last steps how to

implement Continuous Monitoring

Copyri

ght

18

Introduce a central event

handler which …

• Will become your big data

lake

• Will contain events and

alerts ( actionable events )

• Is used for dash boarding

• Trend analyses over time

• Application logging should

contain ARM points or

UUID for traceability

• Can escalate events to

alerts over time by

complex correlation.

• Should feed your IT chain

dashboards for deeper

chain analytics.

• At least 30 days of raw

operational data

First step – Centralize your event data

Data Lake

Infra

Monitoring

User

Experience

Monitoring

Application

Monitoring

Copyri

ght

19

IT Value Chain

Business Application Business Application

Load balancer Load balancer

CMS

CMDB

Populate chain

&

Application dependencies

Employee is boarding a new customer

Confirmation

by post fails

SYMPTOM

SYMPTOM

SYMPTOM

Print queue

offline

CAUSE

Get to know your IT

Value Chains by …

• Use CMS tools for

discovering your IT

landscape with their

dependencies and to

keep configuration

management up-to-

date.

• Giving the DevOps

teams the possibility

to see the business

impact, track down

all specific events,

deepdive and identify

the cause event and

other symptom

events within one or

more related

business

applications.

Second step – Introduce IT Chain Monitoring

Copyri

ght

20

Introduce a analytics system

to ….

• Analyze all your

monitoring historical data,

like events, metrics and

health states for the

machine learning

algorithms.

• These pre-processing will

find patterns to

automatically produce a

model for post processing

on production data

streams.

• After one of the patterns is

matched a prediction by

an early warning alert will

send out.

Third step – Put Analytics on your monitoring

data

Predictive Analytics Engine

EventsMetricsHistorical

Data sources

Prediction

Pre-processing Patterns

Post-processingModel

Produce model

Algorithms

Input

Output

Copyri

ght

21

Fourth step – Review your feedback loop for your implemented

Continuous Monitoring toolchain

IT FactoryIT Delivery

Production

environment

SYSTEM Actor

End users

PEOPLE Actor

IT Management

Support desk

PEOPLE Actor

IT Management Tool chain

SYSTEM Actor

DevOps Engineers

PEOPLE Actor

Delivery Pipeline

Copyri

ght

22

The Life of an Event

CONCLUSION

EVERY EVENT HAS A PURPOSE FOR

EXISTENCE AND DURING HIS

LIFETIME MULTIPLE INTER-

RELATIONS WILL EXIST. THIS

BEHAVIOUR WILL HELP US TO LEARN

AND DETECT PATTERNS

AT LAST BE PREDICTIVE FOR THE

FUTURE

Copyri

ght

23