presented to: by: date: federal aviation administration enterprise messaging system (ems) soa brown...

Presented to:

By:

Date:

Federal AviationAdministration

Enterprise Messaging System (EMS)

SOA Brown Bag #6

SWIM Team

April 14, 2011

2Federal AviationAdministration

SWIM Enterprise Messaging SystemApril 14, 2011

Agenda

• SWIM Current Status

• Plans for Moving Forward

• EMS Architecture Overview

• EMS Messaging Patterns

• EMS Pub/Sub Messaging Overview

• EMS Pub/Sub On-Ramping Overview

• EMS On-Ramping Lessons Learned



Governance Roles and Responsibilities

• NAS Governance – SWIM performs Service-Oriented Architecture (SOA) suitability

assessments in support of the Enterprise Architecture Review Board/Technical Review Board (EAB/TRB)

• Identify potential providers of NAS services

• Programs assessed at investment decisions (IARD, IID, FID)

– EAB/TRB approves authorized providers of SWIM-compliant National Airspace System (NAS) services

• Enterprise SOA Governance– SWIM ensures governance compliance– SWIM will NOT attempt to govern SOA implementations that are

internal to NAS programs



Governance Implications

• Programs will use the enterprise SOA infrastructure provided by SWIM

• Programs will not develop their own redundant enterprise SOA infrastructure

• Programs will meet SWIM-compliance requirements as required by EAB/TRB

• Disputes related to implementation of enterprise SOA will be resolved by the EAB/TRB



Financial Implications

• Enterprise SOA infrastructure costs will be included in the SWIM Segment 2 baseline

• SWIM-compliance costs for NAS services will be included in each program’s Joint Resources Council (JRC) funding request

– Programs will prepare SWIM Provider Provisioning Plan (SPPP) at the Final Investment Decision (FID)

• Joint effort between program and SWIM

• Based on the program’s Final Program Requirements and Architecture



Funding SWIM Services

• General Approach:– SWIM conducted JRC, with F&E

funding requested to develop initial Core Services, followed by “user-funded Operations & Maintenance (O&M)”

– JRC conducted by Service-requesting program; funding provided to SWIM, who builds service with O&M phase “user- funded”

• Requires knowledge of:– Total costs to provide Core Services– Number of anticipated service

consumers– Schedule of need dates for service

consumers• Decision on service development costs

and benefits: – Spread among all consumers? – “First consumer” bears the full burden

of the cost? – NextGen expected to provide FY11

development cost for:• Network Time Protocol (NTP)

• Identity and Key Management (IKM)



Plans for Moving Forward

• Building out initial nodes– ACY (May) & ATL (June)

• Planning future upgrades– Additional Customer-Required Functionality– Additional Nodes (e.g. OKC, SLC)

• Identifying customers– SPPP– Producer Template (ITWS, CIWS, WMSCR, TBFM/TMA,

NNEW, AIM, MAPS)– Consumer Template (Internal and External)– Producer/Consumer Workshops



Service Domain

Tier

Enterprise Domain Tier

Policy Implementation Points

WAN ServicesNetwork Security

Enterprise Domain Tier ESB

Stakeholders – Make DecisionsProcess – Procedures, Metrics,

Control MechanismsPolicies – Decisions Made

Domain Specific Governance

Providing Local Autonomy

NAS-WideGovernance Providing

Central Direction

Infrastructure Service Developers

Service Portfolio Management

EnterpriseService

Repository

Federated Repository

System/Security Admin

Service Level SecurityEnterprise Services

Identity ManagementAccess Management

Service ManagementEnterprise Services

Service Administration

SponsorPrograms

Enterprise Registry/Catalog

Federated Registry

Legend

DEX Service

FTI Service

Service Container Enabled Apps

Legacy NAS System Apps

Separates the IT and integration

infrastructure concerns from domain specific

ATM concerns

SWIM Implementing Program

SWIM Service Container

NAS ApplicationsNAS ApplicationsNAS Applications

Legacy System

Legacy Infrastructure

NAS ApplicationsNAS ApplicationsNAS Applications

TRACON 1

ARTCC n

NAS Enterprise Security Gateway

UESB

NAS Users

External Customers

Interface Toolkit

Managed Access To Enterprise Services

Legacy System Interface

Standards-Based ESB Interface

NAS UsersNAS Users

Enables NAS use of multiple vendor technologies

avoiding lock-in

NAS programs and lines of business are executed by domain experts

DEX

Decoupled using JMS and WS

ARTCC 1

TRACON n

EMS Architecture Overview – DEX Conceptual Architecture



EMS Architecture Overview – DEX Deployment Architecture

DEX SystemManagement

Node

Primary NOCC

FTI WAN

Core ARTCC 1

DEX Messaging Node

DEX SystemManagement

Node

Backup NOCC

DEX Messaging

Node

ARTCC (Eastern Hub)

DEX Access Management

Node

DEX Messaging

Node

ARTCC (Western Hub)


Node

DEX Messaging

Node

ARTCC (Central Hub)


NodeCore ARTCC 1

DEX Messaging

Node

Core ARTCC N

DEX Messaging

Node

DEX Enabled NESG

External FAA Customers (e.g. Airlines, Airports)

NAS

DEX Enabled NESG

External FAA Customers (e.g. Airlines, Airports)

Enterprise Domain Service Bus

Service ManagementService Administration

Service Level SecurityEnterprise Registry/CatalogEnterprise Service Repository



EMS Messaging Patterns

• Publish/Subscribe – implemented using Java Messaging Service (JMS)

• Request/Response – implemented using web services

Primary emphasis has been Pub/Sub messaging services



EMS DEX Pub/Sub Messaging Overview

Producer

Routing Info

CBR

Headerdata

Headerdata

Headerdata

Messaging Service

Subscription Configuration(DEX Navigator)

Content Taxonomy(Content Catalog)

DEX Admin or Consumer

1. Define messages, metadata, and taxonomy

1. Define messages, metadata, and taxonomy

2. Connect Producer for publishing via JMS

3. Configure subscription content for Consumers

4. Connect Consumers and subscribe to assigned topic(s)

5. Route data to Consumer(s) based on currently configure subscriptions

Content Stream

Jump Start Kit

Consumer A

Jump Start Kit

Consumer B

Jump Start Kit



Pub/Sub On-ramping Overview - Producer

Content products & metadata

defined by Producer

Messages, routing attributes and

content taxonomy defined,

Producer queue needs defined

Producer queue(s) configured,

names provided to Producer

Content Catalog updated

by DEX administrator

Optional Dex Jumpstart Kit“On-ramping” Tool

Producer application

connects to DEX

DEX connection information and client

JAR file provided by DEX administrator

DEX Activity

Producer Activity

Producer and

DEX Activity



Pub/Sub On-ramping Overview - Consumer

DEX Navigator

Connection information

& credentials provided


Content list & metadata

accessed by Consumer

Content & topic(s) selected by

Consumer to define subscription(s)

Content & topic(s) configured by

DEX administrator to support subscription(s)

Optional Dex Jumpstart Kit“On-ramping” Tool

DEX Navigator

Consumer application

connects to DEX

DEX connection information and

client JAR file provided


DEX Activity

Consumer Activity

Consumer and

DEX Activity



EMS DEX Pub/Sub Messaging – Lessons Learned

• Producer Push Model

• Enterprise Service Bus (ESB) Interoperability Support

• Web Service Virtualization

• Message Compression

• Message Filtering

• Multi-threading

• Message Batching



Producer Push Model

• Benefits– A common set of security policies can be established and enforced for

accessing enterprise level information– The sharing of all enterprise level information can be monitored from the

Producer service access point to the Consumer access point providing SLA policy enforcement and enhanced situational awareness

– A common approach to providing effective load balancing and highly available messaging for enterprise level data can be leveraged across the enterprise

– Loose coupling with providers– Common interface for providers

ProducerFTI WAN

Content

DEX Messaging Node

Service Bus Processor

Products are pushed to the DEX



SAP

Lo

ad

Ba

lan

cer



ESB Interoperability Support

• ESB has powerful interoperability capabilities– Demonstrated support for JMS interoperability between

different vendors (Oracle, IBM, Progress)

– Demonstrated application level messaging protocol mediation

• JMS to Web Services (WS) – JMS producer sending to WS consumer

• WS to JMS – WS producer sending to JMS consumer

– Message format transformations• JMS – Simple Object Access Protocol (SOAP)

• SOAP – JMS

• Extensible Markup Language (XML) - XML



Web Service Virtualization

• Definition – management of web service endpoints transparently for consumers

• Alternatives

– Run-time Registry, Load balancer, ESB

• Experience to date – Run-time Registry can interoperate with ESB

– Run-time Registry may be overkill (high $$$) given anticipated NextGen scenarios (Weather Domain may be exception)

– Load Balancer on Roadmap

– ESB message flows can be configured with web service endpoints, this is current approach of DEX



Message Compression

• Message compression can be used to increase throughput while lowering network bandwidth utilization

• Compression becomes more beneficial as message size increases (500B ~25%, 10kB ~80%)

• Alternative approaches - Producer compression, DEX compression

• Producer – If messages are small Producer can pack messages and compress

yielding better results

– Consumer would need to decompress and unpack messages

• DEX– For larger messages DEX can compress received messages, route to

consumers, and decompress transparently to consumers



Message Filtering

• Definition – removing or altering the content of a message payload

• Alternative approaches - Producer filtering, DEX filtering

• Producer filtering

– Producer filters data and publishes filtered and non-filtered messages tagged to indicate filtering characteristics to DEX

– DEX routes messages to authorized users per established policy using header attributes

– Use when domain specific business rules govern filtering to keep business domain decoupled from enterprise domain – e.g., Airport Surface Detection Equipment, Model X (ASDE-X) sensitivity filtering

• DEX filtering

– DEX filters data and publishes data to authorized users per established policy

– Use when generic business rules govern filtering – e.g., unconditionally clearing given fields of a message for a class of consumers



Multi-threading

• Messaging throughout can often be adversely effected by message transfer delay (latency)

• This is especially true for acknowledged messages.

• For example, a roundtrip latency of 10 milliseconds (ms) would limit a sequential message transfer using a single thread to no greater than 100 messages per second (mps)

Thread

Message 1 Message 100

1000 ms10 ms 20 ms 990 ms




Multi-threading

• Multi-threading – using multiple threads in a process to achieve more effective utilization of processing resources

• Messaging throughout can be increased to overcome high latencies by using a multi-threaded process that is transferring messages

– Enables message transfers to be overlapped to effectively compensate for the transfer latencies of sending and then waiting on the acknowledgement

– Each thread can transfer messages almost concurrently (minimal input/output (I/O) resource wait time) and in this three-thread case almost triple the effective throughput

– As long as there is available bandwidth, the thread count can be increased to achieve increasing throughput

– The other limiting factor on thread count is processing resources of the host processor

Thread

Message 1 Message 100Message 1 Message 99

Message 1 Message 100Message 1 Message 99


1000 ms10 ms 20 ms 990 ms


Net throughput approaches

300 mps

Thread

Thread



Message Batching

• In order to achieve higher message throughput rates over the Wide Area Network (WAN), a message batching feature may be utilized

• Message batching consists of collecting a configurable number of messages (Message Max) and creating one large message at the DEX that contains these messages for subsequent transmission to the consumer

• There are additional configurable parameters that control the amount of time the message batcher will wait for the Message Max to occur for batching

• There is a trade-off between batch size and latency – large batch yields higher latency

• Once the batched message is received on the consumer end, the messages are unpacked and delivered to the end consumer one at a time making the message batching transparent to the end consumer

• Tests with WAN 24ms round trip latencyTest Case

Publishing Rate (mps)

Message Batch Size

Consumer Receipt Rate (mps)

1 1000 50 788.692 1500 100 1025.983 2000 500 1785.944 3000 1000 2659.635 3500 1000 2839.71



WWW.SWIM.GOV

presented to: by: date: federal aviation administration enterprise messaging system (ems) soa brown...

Documents

swim programs

swim team

swim segment

nas programs

governance compliance

swimcompliance requirements

authorized providers

baseline swimcompliance