dynamic real time complex event processing · 2019-11-26 · data warehouse pda data real-time...

IBM Software Group

© 2007 IBM Corporation

Dynamic Real Time Complex Event Processing

A Department of Defense (DoD)/Ministry of Defense (MoD) Perspective

Real Time Middleware Seminar 2007

Haifa Israel15 April 2007

��

© 2007 IBM Corporation�

IBM Software Group

Agenda� Defining the Challenge – Problem Space

� CEP Consideration � Static Case� Dynamic Case

� Introduction to Real–Time� Why real–time?� What is “real–time?” What does it mean? In what context?� How is it possible to characterize real–time systems?� Real–Time and a programming model

� Dynamic RT CEP� SCA Model� Architectural Consideration

� Use Case Scenario – Military� Review the Challenge


IBM Software Group

DoD/MoD Industry Strategic Goals� To Achieve Net Centric Warfare and Operations, four key domains must intersect

� Physical: Traditional Warfare Domain where forces are moved through time and space. It includes physical assets and IT systems

� Information: Domain where information is created, manipulated and shared. This facilities the sharing of information

� Cognitive: This is the mind of the Warfighter and what drive Effects based Operations

� Social: Is the necessary elements of human enterprise. It is where human interact, exchange information, form shared awareness and understanding to make collaborative decisions

Social DomainCollaborative Decisions

Cognitive DomainCognitive Advantage

Information DomainInformation Advantage

Shared Awareness

Physical DomainForce Advantage

Position advantagePrecisionForce

CompressedOperations

Speed and Access

Plan, Organize, d\Deploy,Employ and Sustain operations

ConveyedCommander’s

intent

NCWO

InformationAge

Warfare


IBM Software Group

DoD/MoD Industry Strategic Goals 2010 :Net–Centric Operations Warfare (NCOW)

� Has become the dominate initiative since September 2001� Resulting from changing battlefield; asymmetric warfare is the order of the day, not classical military to military clashes

� Processes and policies must be recast and redesigned to drive effects–based operations� Need to contain adversaries to prevent their ability to conduct operations.

� Underlying this is the need for diverse systems and military resources to have the ability to interoperate and share information, resources, and services in a secure and deterministic manner.

� The key governing principles are:� Fight First for Information Superiority

� Access to information: Shared Awareness

� Speed of Command and decision making

� Self Synchronization

� Dispersed Forces : Non–contiguous operations

� Demassification (use information to achieve desired effects with a smaller force in a specific geography)

� Deep sensor reach

� Alter initial conditions at higher rates of change

� Compressed operations


IBM Software Group

Challenge – Defining the “Problem Space”� Inability to manage latency and determinism across widely dispersed nodes for predictable

end–to–end processing

� A highly fluid and dynamic ecosystem: fixed event–producing and event–consuming nodes coupled with transient nodes, have bounded availability

� How to first tolerate, then exploit the value of these transient nodes into the enterprise, achieving time critical operations / situations

• Dynamically discover nodes and services• Register (Pre-register at initialization, or leverage tooling for on–the–fly registration)• Prioritize with respect to other transient nodes• Ensure temporal and causal interlock• Compose and integrate into an appropriate CEP flow / domain• Dynamically reconfigure run–time behaviors to support the workflow• Recognize duration and availability of boundaries / limits• De-allocate transient node and dynamically reconfigure the network

� Challenge: How to think / view the problem at different levels of the stack (vertical versus horizontal)


IBM Software Group

Rivet JointRivet Joint

AWACSAWACSGlobal Hawk

Multi-INTGlobal Hawk

Multi-INT

Sensors /Shooters

Sensors /Shooters

GPS+GPS+

SBR

SBIRS+SBIRS+

UCAVUCAVPredatorPredator

TankerTanker

RT Dynamic CEP – Integration in the LargeRT Dynamic CEP – Integration in the Large

MC2AMC2A

FIAFIA

AirliftAirlift

TALCE/MSTTALCE/MSTCAOCCAOC

AMEAME

AFFORAFFOR--FF

ARGUS ARGUS FAMILY UGSFAMILY UGS

ASOCASOC

CRC/BCSCRC/BCS

TACPTACP

WOC/EOCWOC/EOC

AF DCGSAF DCGS

SOFSOF

TACCTACC

SPACEAF SPACEAF AOCAOC NIMANIMA

NSANSASPACESPACE

C2C2

WeatherWeatherC2C2

AF DCGSAF DCGSCAOCCAOC

AFFORAFFOR--RR

IOSAIOSA

Joint Force with common situational awareness, common operating picture, informed decision making

Land ComponentLand Component

Maritime Component

Maritime Component

� Massive numbers of sensors of all kinds�Unstructured data

�Fixed as well Transient event producers and consumers

� Massive numbers of “actuators” of all kinds

� Massive (potential) connectivity�Many partly, or rapidly changing dynamically–connected edges

� Many human beings, organizations in decision loops

� A nested collection of classic control systems..

� An Internet-Scale Control System that is a large stochastic process


IBM Software Group







IBM Software Group

Event Processing Agent : Basic Capabilities

EPASource Consumer

EPA

Source

Source

Support Components�Registration�Discovery�Monitoring�Linkage

Consumer

Event Producer or Sources: (Sensors, IT generated, Business Process/application output)

Event Consumers: Other systems, EPA’s, Application/Business processes

Event Processing Components: (EPA’s and endpoints that could be applications as part of an overall business process)

Supporting Components: Event Processing Network, External service for data enrichment, Monitoring and management (i.e. Event Broker)

� First generation event–driven systems: based on predefined events, node–to–node relationships

� Performed relatively simple event pattern detections with various transformation capabilities:

• Aggregation• Translation• Composing• Splitting

� Systems had fixed end–to–end Event Processing (Complex or Simple) workflows

• They were data deterministic• They were not necessarily latency deterministic

� Not standards based

• None were defined and vendors created their own� Were predefined as part of their respective registration and system development processes


IBM Software Group

Event Processing Agent : Second Generation� Second generation event processing technologies must be based on a set of industry defined and

adopted standards and definitions.

� Technology convergence has pushed the event processing community to realize Standards are critical.

� Will still be static albeit more capable and standards based.

� May or may not support real time ( Latency determinism).� Unable to dynamically discover, integrate, and enable event–producers and event–consumers

that are not fixed in a specific infrastructure/enterprise.

Invoke SOA service

EPA = Event Process Agent

EPAEvent ChannelSource

Situational Awareness

Business Sub-process

Activity

Service (Data Retrieval)

EPA

Source

Source

EPAComplex

Event ComplexEvent

ComplexEvent

Enterprise Service Bus

�Registration�Discovery�Monitoring�Validation�Mediation�Linkage

�Validation�Enrichment�Transformation�Routing�Error handling

Event Stream

Event ChannelEvent Channel

Event Channel

Event Channel

Point-to-PointPublish/Subscribe

© 2007 IBM Corporation��

IBM Software Group

Event Processing : The Future

Command & Control

FixedNode

TransientNode

FixedNode

FixedNode

TransientNode

TransientNode

Event Processing BasedSOA Enterprise

� Future EP enterprises will need to support data and latency determinism across a continuum of performance capabilities for fixed and transient event nodes.

� Abstract the complexity of Event Processing into a RT Enterprise Service Bus (ESB)

� Augment and enrich predefined fixed Complex Event Processing (CEP) flows with transient event producers and consumers via dynamic discovery

� Pre-register event sources for future use in a CEP process flow

� Enable and support rule–rich and processing–rich, intelligent–event–agent nodes that can recognize newly active event nodes and reconfigure rules engine

� Leverage caching, knowledge modeling to accelerate system response

� Couple node–to–node and end–to–end Quality of Service (QoS) policies tightly

� Support robust, end–user tools

• Enable on-demand, run–time generation and registration of new event nodes• Enable workflow composition


IBM Software Group

Basic Event Atomic Structure

MediationRules

Events(1)

Out Comes

Event Process(EP)

Events (n)


IBM Software Group

Event Processing Agent EPA

F(r)

CEP(i=1)

)],..1(),(),([ xnxmniE ��

)],..1(),1(),([ xnxmiE ��

CEP(i=n)

ArgumentsPassedxnxm

TrackHistoryCausalEventiz

IDEventi

ZuluStampTime

txExnxmiziE

dtRftxECEP

_),...1(___)(

_)()(_

),()],..1(),(),(,[

)(),(

�

�

�

�

�

��

�

�

��

Complex Event Processing Agent EPA Message Arguments

F(r) = EPA Rules


IBM Software Group

Eve

nt B

roke

r

Event Topic (1)

Example of Static Complex Event Processing Flow

Validation

QoSB � CLatencyJitter

FilteringA � BA ��B

Enrichment

ComputationalEnhancement

PatternRecognition

Transformation

Fusion

Aggregation(A + B +C)

Routing

Pass Through

Hold and Store

Event Driven ESB

F

Event Topic (2)

Event Topic (i)

CEPUsers

CEPUsers

EPA 1

EPA 2

EPA 3 EPA 4

F

F


IBM Software Group

F(r1)

EPA1; First Order EPA2: Second Order

)],..1(),(),1(),([ xnxmiziE ��)],..1(),1(),([ xnxmiE ��

)],..1(),(),([ xnxmniE ��

CEP(i=n)

CEP(i=1)

CEP(i=n+1)

F

F

T

T T

F

F(r1)

F

F(r2)

F(r2)

Select alternate Rules set

Based on Transient persistence

Select alternate Rules set

Based on Transient persistence

Example of Dynamic Complex Event Processing Flow

Dynamic RT Event Driven ESB


IBM Software Group

Dynamic RT CEP Process Flow

T=0

Pre-registration

Transient Event Node Pre-registration parameters

� Event ID number

� Events QoS Policy

� Interface requirements/protocols

� Assignment to a predefined Community of Interest (COI)

� Capability (process, transform, route, enrich)

� Relationship to other fixed or transient assets in COI (Inputs, Outputs)

� Discovery when asset interrogates network with proper authentication, access rights and security level

Discovery/registration

Dynamic� IP Address Interrogation ( Are you

there )

� IP Address Authentication ( Do you have permission )

� Search for pre-registration event component/dossier

� Gather Temporal Information (Geo-spatial, duration )

� Compute Area of Operation (AoR) assignment

� Start Duration clock

� Notify CEP Domain Community of Interest (COI) of transient node(s) availability

� Dynamic reconfiguration of node-to-node or end-to-end Complex Event Processing flow QoS with special attention to alternate rules set within Event processing agents

Manual� Manual Registration of undefined

assets:

� Authentication, Access privileges

� Interface/protocol definition

� Assignment to COI and relationship

� Capability

� Dynamic assignment to a COI’s

Active Transient Event(s) enablement

� Notify event agent of Transient event availability/activation

� Switch rules engine

Active Transient Event(s) dis-enablement

� Notify event agent of Transient event Termination/availability

� Switch rules engine

Reconfiguration CEP Composition

� Define the processing states for an event

� Define acceptable outcomes

� Define anomalies, and response scenarios


IBM Software Group

Real-Time Event Driven Component

State - (State full/Stateless)

Import/Export Parameters

Data Type

Function – Internal process or action F(r)

FundamentalCapability

New function for Dynamic Real-Time

�Event Flow

�Process Flow

�Mediation Flow

�Individual QoS�End to End QoS�EPA Node rule selection based on transient input activation period

Component Component

Component

Component

Real-Time Event Driven Composition of

Component

Real-TimeRun-time

Deployment

RT Event Driven Architecture SCA Model Abstraction

Realization

Persistence time

Function F(r) is selectable based on transient input discovery/activation period

Quality of Service (QoS) Policy


IBM Software Group





� Use Case Scenario - Military� Review the Challenge


IBM Software Group

ESB

Real-time in event-driven architectures Where does it fitEvent Driven Architecture• Augments transactional business process and

application server within SOA framework

• ESB enriched to gather and detect significant patterns (temporal, spatial) in event data

• Delivers detected events to relevant processing applications – in both push and pull modes

• Significant advance beyond current request /message at a time routing and delivery by ESB

Sensors, RFID readers…

Control nets,actuators

Application history, data warehouse

PDA data

Real-time Events

• EDA supports the integration of real-time, near real-time and enterpriseapplications

• Timeliness and dependability new,required QoS in nodes and in the ESB toprocess events in time

• Time domains differ across EDA layers

• e.g. industrial automation, voice/mediaswitching, network-centric warfare

< 10ms

1-2s

non-RT

Sensing Weapons

Tracking

Command and control

Intelligence Ship systems

Ship applications

Provision-ing

Force coordination

Each ‘domain’ has QoS attributes setting granularity e.g.

• ship applications – enterprise-like, seconds , transactional

• command & control, intelligence, ship systems – soft RT, 100ms-1s

• tracking, sensors, weapons – harder RT, 10-100ms

• leaf components – devices, sub 10s hard RT response

Track objects

Assess / authorize


IBM Software Group

Real Time Systems Cover A Broad Deterministic Continuum

Embedded

System

Enterprise

Ecosystem

Em

bedd

ed

RT

devi

ces

Onboard Server-class RT Command & Ctrl

Onboard Server-class RT Command & CtrlESB

Events are everywhere and their interaction is a function of your frame of reference ( Big picture enterprise to embedded) and your needs.

They can happen at multiple levels simultaneously

�Different time epochs�(non to hard Real Time)�System Type

�Closed- No external inputs�Open - Bounded�Open - Un-bounded

We are here today

We are building this


IBM Software Group

What do we mean by “Real–Time,” exactly?� Repeatable, consistent behavior

� Causes of non-determinism have been minimized or eliminated

� Is a property:� Guaranteed and predictable deterministic completion of an “operation” to event in situ

� Is not necessarily fast, normally an indication of low latency:� Often appears in conjunction with latency requirements� Entire environment has factors for additional consideration

� Recognize that the deadline is guaranteed with predictable results – key!� If not achieved, requires some other action be taken, that some exception be raised

� No single answer� May be “hard” in the sense that violation of a timing constraint is considered an application failure, or

“soft” if timing constraints are simply stringent performance goals

� Consequences for missing a timing constraint may vary in severity (e.g. miss on Service Level Agreement, lost revenue, damage to property, danger to human life)

� Timing windows may vary in magnitude (e.g. from microseconds to seconds)

� Not simply about better performance; About greater predictability of performance� Maxim: “Real–Fast is not Real–Time”� Corollary: “Real–Slow is not Real–Good”� Challenge: strike the right balance between predictability and overall performance


IBM Software Group

� Hard �The system must meet its timing constraint on 100% of operations

� A single failure is considered a total application failure

� Soft�Meeting a timing constraint within a predefined accepted performance

parameters (i.e. within 85% or better)

� Hard real time - historically a niche market �Specialized hardware, RT O/S, RT software stack running on dedicated

uniprocessors

�RT Application code such as Assembly, C, C++, Ada

�Typically low latencies requirements - microseconds

What do We Mean by “Real Time” Exactly? Cont’d


IBM Software Group

From Seconds to Nanoseconds…

10 s

1 s

100 �s

10 ms

100 ms

1 ms

10 �s

STRATEGY

TACTICS

COORDINATION

ACTUATION

SENSING

MODULATION

SIGNALING CUSTOM HARDWARE

PE

RC

EP

TIO

N

RE

AC

TIO

N

CO

GN

ITIO

N

Incr

easi

ng s

oftw

are

com

plex

ity

Software complexity vs. Time domain


IBM Software Group

What are Real Time Goals?

� Program must complete a task in or by a certain time deadline and with a predictable degree of certainty (and, latency is additive)

� Time deadline (Time resolution or granularity)� 300ms GUI (graphical user interface)

� 100ms Network-based transaction

� 10ms multimedia

� 1ms “Real Time” (often physically-based) tasks

� Degree of certainty� 90% Interactive response (Word processor)

� 99% Soft Real Time (Web service)

� 99.9% Firm Real Time (multimedia, telecom)

� 100% Hard Real Time (flight control, radar/sonar loop)


IBM Software Group

Metronome Programming Abstractions

10 s

1 s

100 �s

10 ms

100 ms

1 ms

10 �s

RealTimeThread with Heap

NoHeapRealTimeThreadwith Scopes


IBM Software Group

Using Java for Real-time Development

� Incremental path to real-time Java

RTSJ + (NHRT) Non-Heap Real-Time

RTSJ (heap) + Realtime garbage collection

Std Java + Realtime garbage collection

Std Java + Object Pooling

Standard Java

Realtime (<5ms)Realtime (10sec – 5ms)Non-Realtime

Direction of faster response time


IBM Software Group

WRT Architecture

Javajar

J9 VM Real-time GC

Metronome

Java SE 5.0 Libraries

Real-time feature (“-Xrealtime”) Standard J9 component

X86-32 Opertron (4-way SMP)

Real-time Linux (Based on RedHat 4.2)

Bind

Boundjar

AOT

Compile Real-timeJIT

Real-time

library


IBM Software Group

Websphere Real Time Product Components

J9 VM Metronome

Real-time GC

Java SE 5.0 Libraries

Standard J9 component

Real-time Linux (Based on RedHat 4.2)

Real-timeJIT

Real-time Java

Libraries

Java Run-time Environment (JRE)

Software Development Kit (SDK)

jxeinjar

RT- Aware

Debugging tools

RT Class

Source


IBM Software Group

Linux and Open Source Matter to Raytheon

Key Benefits � Open Source Solution that is on track to be adopted by the Linux Mainline�Context switch latency under 25 µs (99.9999% under 20 µs) �Open Real Time Stack: RT Linux- RTSJ & RT GC RT Java- x86 Blades�Reduced Risk and Reduced Total Cost of Ownership

Solution�Real Time Linux – Led by the IBM Linux Technology Center and built on the work of Red Hat and Open Source Community�IBM System x and BladeCenter based solution�Fully preemptive kernel, reducing critical path latencies�Priority inheritance enabled kernel and userspace locking

Challenge�Build a Real Time Linux Operating System that would compliment the RT Java to meet the performance demands of the DDG-1000 program while working with the Linux Community to mainline the enhancements.

DDG 1000Zumwalt Class


IBM Software Group

IBM’s RT Run-Time Execution Stack for J2SE Today

RT Programming ModelC++ & WebSphere Real Time Java

NetworkUDP/IP TCP/IP

HardwareOpteron x86-32 4-way SMP Blades,

RTOSRT Linux (RHEL V4U2 with RT patches)

RT MessagingDDS V4.1

INFORMATION MANAGEMENT•ANTs IMDB

Development•Tuning Fork


IBM Software Group

Quality of Service Parameters

Defines maximum worst case acceptable latency delay in message delivery from a publisher to a subscriberLatency Budget

Define persistence of transmitted data/message availability to newly joining subscribers. Three possible setting are available:

�Volatile-No past data Kept

�Transient- System will keep certain number of samples determined by resource limits and History Parameters

�Persistent- System will store all past message to either in-in system memory or a database

Durability

Specifies no missed data so data must be retransmitted. It has two sub-parameters:

Reliable- Data retransmission will occur until data is received or until time out occurs

Best Effort- (Default) no data retransmission will occur

Reliability

Specifies how many data samples will be stored for later delivery and is tied to Durability QoS . Can be set to keep last N samples or AllHistory

Minimum rate at which a

�Data Writer or publisher will send data

�Data Reader or Subscriber is willing to wait for data

Deadline

Establishes RT Threads priorities from 1- 28 with 1 being highest:

For RT Java Thread Priorities are as follows:

�NHRT Threads- 1 to 8

�RT Java Threads- 9 to 20

�Regular Java Thread- 21 to 28

Determines if a message/data entity is still alive and is tightly tied to the Life Span QoS. One of three values can be set:

�Automatic: Alive signal sent automatically based on Life Span setting or default is infinite

�Manual by Participant: Entity is considered alive if other entities in Domain are

�Manual by Topic: Publisher is alive if at least one instance of its topic has been asserted by the application

Maximum duration that any one message is valid. After such time data is removed form caches, transient history queues or persistent history queues

Feature

Thread Priorities

Liveliness

Life Span

Destination order

Parameters


IBM Software Group





� Use Case Scenario - Military� Review the Challenge


IBM Software Group

High Level SOA Representation for Dynamic RT CEP

ProcessingRT ESB

Transport

Registry

Orchestration

Aggregation

TransformationMessaging

RT QoS Policies

Services Level Agreements

Interfaces

DataDynamic Discover

Admission Control

Create

Compose

Enrich

RetireRT SOA Based EDA

Monitor

&

Manage

SLA

Alerts

Netw

ork Devices

Multilevel

Security

Com

munities of Interest

Attachm

ent Policies

Label Policies

SOA Composed Business Fabric

Multilevel Secure Presentation Layer

SATCOM

Manual Event

On-Demand

Registration

Composition

Monitoring

Deploy

Terrestrial Wireless


IBM Software Group







IBM Software Group

Moving from Deconfliction to Horizontal IntegrationMoving from Deconfliction to Horizontal IntegrationMoving from Deconfliction to Horizontal IntegrationMoving from Deconfliction to Horizontal IntegrationTime sensitive targets must be shared, distributed, Time sensitive targets must be shared, distributed,

and integrated horizontally and integrated horizontally …… to one and all simultaneously to one and all simultaneously ��

FindersFinders

ShootersShooters

DecidersDeciders

ConnectorsConnectors


IBM Software Group

��

Node: ISR1

��

��

��

Node: ISRn

��

��

��

Node: C21C

onne

ctor

Infr

astr

uctu

re

��

Node: C22

��

Node: S1

��

Node: S2

��

Node: S3

��

Node: Sn

Dee

p S

enso

r an

d H

isto

rica

l dat

a R

each

��

��

��

��

��

Near Real-Time Soft Real-Time Hard Real-Time


IBM Software Group

Scenario Description:Unknown Aircraft enters NO-FLY ZONEPre-setup� Air Operations Center (AOC) controls several aircraft – fighters and an AWACS Early warning aircraft

(all are transient nodes)� Command has deployed a defensive missile battery (transient node) forward

� Discover, assimilate transient nodes into AOC domains, based upon persistence in Area of Operation (AOR)� Sensor� Track� Weapons event processing

� Pre–register parameter (s)

Track to Engage Process� Several sensors track unknown target, send track updates; Transients send their location updates to AOC

� Update and compute each transient node’s geo-spatial location for intercept

� Sensor sends:� Alert trigger event� Alert track event (containing the target’s location, altitude, speed and classification)

� Receive alert track and an IFF identification as Foe� System decides to attack the threat, and an intercept asset is sent

• Simple solution: Static assignment according to target location• Complex solution: Dynamic asset assimilation queuing–tasking

� When an intercept asset event and an alert confirmation are received, the intercept missile is fired


IBM Software Group

Air Operations Center (AOC)Air Operations Center (AOC)

No-Fly Zone

Unknown

��!�� "��

��# �$�%��&� �� '�"��

Sensor 1AWACS Aircraft

Sensor 2Forward deployed Radar site

Sensor 3Command Center Radar

CommunicationLink

InterceptAircraft 1

Intercept Point

T

T

F

F

T

InterceptAircraft 2

T

Mobile Weapon System 1Forward deployed


IBM Software Group

High level Dynamic RT Event Processing FlowUse Case

Sensor 1

Sensor 2Sensor 3

EP 1

TrackData

Sensor 1

Sensor 2Sensor 3

AlertTrigger

EP 2

A1orA2orA3

E[Alert trigger (Ax)]

��

)321( orAsorAsAsAT ��

)(ATIFFAT �E[Alert Track]

EP 4

Fire Intercept Missile

EP 3 EP 5

ASSET 1

ASSET 2

ASSET n

AssetsThat canIntercept

(FINDERS)

IFF(AT) Trigger

E[Alert Confirm]

Compute Intercept PointAssign asset

E[Alert Track]

E[Intercept Asset (Asn)]

(DECIDER) (SHOOTER)

Total Time from Detect to Engage = 10 seconds

Microsecond ESB

Millisecond ESB

Second(s) ESB

T

T

T

T

TF

F

FF

A2orA3

)32( SSAx ��


IBM Software Group

Boost to Terminal Phase Community of Engagement (COE)

Boost

Ascent

Separation

Mid-Course

MIRV

Terminal

ReentryVehicle

Decoy

Battle Battle Management/CommManagement/Command, Control & and, Control & Communication Communication (BM/C3)(BM/C3)

SBIR�HEO�LEO�GEO

Terminal High Altitude

Area Defense(THAAD)

Patriot Missile PAC-3

SM-3

KEITHAAD

PAC-3GBI

Aegis CruiserAegis Destroyer

BMEWSCobraDane

Ground Base Interceptor (GBI)FT Greenly Alaska

Vandenberg AFB CAKinetic Energy Interceptor

(KEI)

Airborne Laser(ABL)

Sea BasedX-Band

COE(Boost)

COE(MID)

COE(Terminal)

MobileX-Band

STSSDSP


IBM Software Group

Problem Domain at a High level

� The primary components of what we shall call a Community of Engagement (COE) consist of Sensors, Command and Control and Weapons. These can be further characterized and classified according to:

� Geospatial location

� Flight Profile that they are specifically designed to engage to missile (i.e. Boost & Ascent, Midcourse or Terminal phase)

� Fixed or Mobile (i.e. Aegis Ship, Low earth orbit/High earth Orbit Satellite, Mobile X-band Radar (Land or Sea based) or Missile battery) asset.

� Each of the assets have differing internal and external Quality of Service (QoS) capabilities which must be harmonized and choreographed together to form a COE that meets a specific Service Level Agreement based on the three core attributes above.

� Once a detection event has occurred, our Missile Defense IT Ecosystem must be able to dynamically create a series of COE that track the missile until it is destroyed passing information and getting ready the next COE based on Missile flight path phase as a backup in the event the current COE fails to destroy the missile. In this way COE’s are created, made ready, queued and tasked in event order so as to maximize the probability of kill.


IBM Software Group

Problem Domain at a High level - Continued

� When creating COE’s, they will consist of a number of both fixed COE’s where by the IT ecosystem has established QoS and SLA and mobileassets that need to be discovered and dynamically configured into a COE’s work flow process including QoS policies and SLA’s.

� Herein lies a critical architectural design point with regard to real-time latency determinism. Can a geospatially separated set of services some grouped into a COE – Fixed and others that are added dynamically be auto-configured and provide a deterministic latency performance/predictability that satisfies that COE’s flight profile. Typically the three critical inhibitors to determinism are:� Bandwidth/Communications

� Memory

� CPU processing


IBM Software Group

Problem Domain at a High level - Continued

� With these three key parameters (Bandwidth/Communication, Memory, CPU processing) how do we architect an SOA/ESB IT infrastructure that can meet the Deterministic latencies required to intercept the Missile.� However, if we examine the end to end Missile engagement process it is a very large

Stochastic Process whereby there is only a probability of achieving RT Determinism

� The most critical constraint is provision in the face of either high data ingest or faults

� How do we degrade gracefully in this situation…. How do we trade off Urgency of processing versus importance

� How will your system perform in the presence of overload and resource starvation ( Shed Load, lockup, etc.)

� It is not good enough to just reduce priorities or make something a higher priorities, Resource management and provisioning play crucial role here this goes for memory, CPU and BW). Also not all processes can be arbitrarily stopped some provide critical admin and support services that must continue to execute otherwise we run the risk of catastrophic system collapse

� This problem is not unlike a futures market whereby you ensure against risk by over provisioning

� We work well on a per single node bases but when executing multiple JVM’s on a single CPU resource provision and scheduling become more complex.


IBM Software Group

ESB deployment Patterns Which one is best for BMD

ESB

SRVC

SRVC

SRVC

SRVC

Global ESB/SOA

ESB

SRVC

SRVC

Directly Connected ESB’s/SOA’s

ESB

SRVC

SRVC

ESB

SRVC

SRVC

Brokered ESB/SOA

ESB SRVC

SRVC

ESB

SRVC

SRVC

ESB

ESB

SRVC

SRVC

Federated ESB’s/SOA’s

ESB

SRVC

SRVC

ESB

SRVC


IBM Software Group

Questions?

dynamic real time complex event processing · 2019-11-26 · data warehouse pda data real-time...

Documents