insights for success - ottawa, may 2013 david webber, information architect, oracle public sector -...

<Insert Picture Here>

Insights for Success - Ottawa, May 2013

David Webber, Information Architect, Oracle Public Sector

- Information Transparency

- Civic Engagement

- Information Collaboration

- Innovation

The following is intended to outline Oracle general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.

The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Disclaimer Notice

Slide 2

© 2011/12/13 Oracle Corporation – Restricted Distribution 3

Agenda

• Improving Transparency, Performance, and Enabling Civic Engagement Through Intergovernmental Collaboration and Innovative Technology– From the customers’ viewpoint, David will share recent NIEM*

successes, along with the technical and political challenges along the way. Being both a strong proponent of open source and having developed a number of world leading solutions himself prior to joining Oracle, he presents an objective perspective of what it takes to effectively leverage SOA to achieve intergovernmental collaboration objectives and successfully roll out new services

– Open Data; what you need to know– Open Source; what you can deliver today– Innovations; what we are working on

• What this all means: better services and economic opportunities for citizens

*NIEM – National Information Exchange Model


Technology Landscape

•Open Data– Open Data Introduction, Solution Architecture and Tools

• Information Alignment– Dictionaries - Key Technology and Foundation Pillar

•NIEM Development Life Cycle– NIEM Concepts, Tools and Adoption, SOA

•Summary– Opportunities for Government



Vision and Potential; EU and Japan

5

◆ Not only further development of IT industry, initiatives seek to create…

① New business created by the integration of IT & data and existing industries

② New business created by the integration of different industries through IT & data 　

As real world data is added, the amount of available information

increases significantly

Expanding into multiple

industries and business areas

New Business using IT & data

New Business created by the integration of IT and

existing industries

New Industries created by the integration of different

industries through IT and data

NarrowIT Industry

NarrowIT Industry

AutomobilesAutomobiles

Construction Equipment Construction Equipment

Medical Equipment

Medical Equipment

EnergyEnergy

RetailRetailRobotsRobots HealthcareHealthcare

AgricultureAgriculture

Energy×Automobiles×Transportation System

Energy×Automobiles×Transportation System

Healthcare×AgricultureHealthcare×Agriculture Robots×Retail×City PlanningRobots×Retail×City Planning

Source: http://semanticommunity.info/@api/deki/files/21578/BrandNiemann02122013.pptx

http://semanticommunity.info/@api/deki/files/21578/BrandNiemann02122013.pptx


Japan – DATA METI Plan

As part of approach to develop “Leading-Edge Integrated Industries”, METI intends to actively release public data in formats that facilitate reuse under

the clear set of reuse rules.

＜ Stimulate the Economy ＞

ProvideRaw Data

・ Edit & Process the raw data 　　　　　　　　　　　　　　→ Create additional value

・ Develop Applications

・ Create useful Websites

・ Build business online systems

・ Develop Information Service

Citizens

Business Operator

Revitalize economic and

industrial activities

METI

Deliver Better

Service

Examples of Public Data

Geological Map

Economic Statistics

Source: http://semanticommunity.info/@api/deki/files/21578/BrandNiemann02122013.pptx

http://semanticommunity.info/@api/deki/files/21578/BrandNiemann02122013.pptx

USA - Background

"Open Data is the new default…

anywhere, anytime on any device,

and everything should be APIs"

Steven VanRoekel

Federal CIO – Whitehouse Digital Strategy Architect

Behind the hype and technology what is really being achieved?

Along with its Open Data Policy, the White House unveiled May 9

Project Open Data, a GitHub-hosted tool kit it hopes will be the

living, breathing implementation arm of its policy

7

http://www.whitehouse.gov/sites/default/files/omb/memoranda/2013/m-13-13.pdf

http://project-open-data.github.io/

© 2012/13 Oracle Corporation – Restricted Distribution 8

Decoding the Digital Strategy

•What exactly are Open Data APIs?

•How can Oracle solutions support them?

•What does a solution architecture look like?

•How does this align with NIEM*?

•What technology pillars and tools are there?



Open Data APIs explained

•A way, via the internet, to securely deliver

information between entities and systems

•API – Application Programming Interface

•Security is supported along with roles and

permissions for requestors, e.g.

– Public information – such as road works, health services, voting

– Private information – patient data, student data, company data

Example – Chicago Pharmacy Vaccines Search

http://www.verifyxml.org

1. Socrata data extract

2. MySQL database

3. Glassfish

4. Open-XDX

5. Prime Faces

6. Hosting site

Combines: rapid development paradigm; open data approach;

open source tools; RESTful web services; community based

resources and delivery; NIEM-aligned information feeds.

10


Oracle Open Data Solution – Open-XDX

•Open-XDX is an all new component designed to deliver

rapid Open Data APIs

• Integrates into Oracle Fusion Middleware foundation

and Database products

•Uses XML configuration templates and code-less

methods and open source NIEM tools

•Allows plug-and-play delivery for Oracle customers and

applications


Example – Military Applicant Onboarding

•Need to examine the applicants health history and particularly prescriptions

•1 in 5 high schoolers today are abusing prescriptions•Drug use leads to at-risk military personnel from in-

theatre adverse reactions and post-traumatic stress•Nationally States are looking to share pharmacy

records to reduce prescription abuse, pill mills and rogue doctors

• In California 80% of prescriptions are for non-residents•DOJ / BJA PMIX program – Prescription Medication

Information eXchange – using NIEM


Requisite Onboarding Flow Process


Medical Authority Role

Combines: SOA; BPM; Semantic tools; Secure web services; community

based resources and delivery; NIEM-based information exchanges.

PMIX Background

Scale of the exchange challenge

15


Integrated Location Intelligence

Draw line around an area of interest and highlight all cases in area, and drill into details of a specific incidents.

Location information integrated with BI to allow analysis of incident patterns, geographical hot spots..

Example – Government Financial Transparency

http://www.checkbooknyc.com/spending_landing/yeartype/B/year/114

Coming soon: PeopleSoft data integration via Open-XDX APIs…

17

Sharing Source Code - GitHub

https://github.com/VerifyXML

18


Applicability to Government Challenges

• Cost of building information exchanges

– Reduction of development sunk costs through minimizing coding tasks

– Allowing rapid prototyping and proof of concepts (agile development)

– Flexibility - dynamic adaptable templates instead of rigid fixed code

– Reuse – templates for common systems / solutions easily shared and adapted

• Government transparency and public information sharing

– Structured content from data stores – e.g. election results reporting, monthly cost

reports, scheduled road maintenance; obvious metadata and semantics

– Simple search requests on data – e.g. part numbers, codes, licenses lookup

– Not intended for unstructured and textual content document delivery

• Practical real world data sharing anywhere

– Direct simple tool with short learning curve and plug and play deployment

NIEM + INFORMATION ALIGNMENT

Dictionaries - Key Technology and Foundation Pillar

NIEM Past and NIEM Future

Lessons Learned


Who steers NIEM currently?

Founders and Voting Members• Dept of Justice• Dept of Homeland Security• Dept of Health and Human Services

Ex-Officio Members• Global Justice Information

Sharing Initiative• Office of Management and Budget• Program Manager, Information

Sharing Environment (ISE)• NASCIO

Partners• Terrorist Screening Center• Dept of Defense / Dept of Navy• Dept of State, Consular Affairs (invited)

DoD NIEM Adoption NIEM military domain “will support development of information exchange specifications across the full range of

military operations.”

NIEM will be used unless component can show compelling reasons

DoD Chief Information Officer Teri Takai :

Lt. Gen. Mark Bowman, Joint Chiefs of Staff CIO/J6

22


National Information Exchange Model (NIEM)

Centralize - Joint DOJ / DHS / HHS program + DoD

1 Created a centralized organization to promote standardization of information exchange for cross jurisdictional information sharing.

NIEM’s governing structure is comprised of Federal, State, Local, Tribal and private organizations. NIEM is managed at an executive level by the Department of Homeland Security (DHS), Department of Justice (DOJ), and Department of Health and Human Services (HHS) + DoD

24

FEDERAL AGENCY COMMITMENTS

AGENCY USE OF NIEMDepartment of Agriculture Committed to UseDepartment of Defense Committed to UseDepartment of Education Committed to UseDepartment of Energy Committed to UseDepartment of Health and Human Services Committed to UseDepartment of Homeland Security Committed to UseDepartment of Housing and Urban Development Committed to UseDepartment of Justice Committed to UseDepartment of Labor Committed to UseDepartment of State Committed to UseDepartment of the Interior Committed to UseDepartment of the Treasury Committed to UseDepartment of Transportation Committed to UseDepartment of Veterans Affairs Committed to UseEnvironmental Protection Agency Further Evaluation RequiredGeneral Services Administration Committed to UseNational Aeronautics and Space Administration Further Evaluation RequiredNational Archives and Records Administration Committed to UseNational Science Foundation Committed to UseNuclear Regulatory Commission Will Not UseOffice of the Director of National Intelligence Committed to UseSocial Security Administration Further Evaluation RequiredGeospatial Line of Business Will Not UseGrants Management Line of Business Further Evaluation RequiredFinancial Management Line of Business Committed to UseHuman Resources Line of Business Committed to Use

The NIEM FrameworkNIEM connects communities of people who share a common need to exchange information in order to advance their missions, and provides a foundation for seamless information exchange between federal, state, local, and tribal agencies. Much more than a data model, NIEM offers an active user community as well as a technical and support framework.

Support FrameworkTechnical FrameworkCommunity

Formal Governance Processes

Online Repositories

Mission-Oriented Domains

Self-Managing Domain Stewards

Data Model

XML Design Rules

Development Methodology

Predefined Deliverables (IEPD)

Tools for Development and Discovery

Established Training Program

Implementation Support

Help Desk & Knowledge Center

The NIEM Data Model

NIEM’s data model is a set of common, controlled, and approved XML data structures and definitions vetted through the Federal, State, Local, Tribal and Private Sectors.

Data elements are organized into core and domain-specific components

Core components are used by multiple

domains and can be described by structure,

semantics, and definition universally

Domain-specific components are

continually updated by subject matter experts that are actual NIEM

participants and industry experts for

their particular domain

NIEM Naming and Design Rules (NDR) specify how each of

these components are defined and utilized

NIEM’s Domains View

NEWEST DOMAINS

NIEM Government Resource

Management Domain

NIEM Health Domain

NIEM Human Services Domain

DOMAINS IN PLANNING

NIEM Agriculture

NIEM Education

NIEM Transportation

27


NIEM Past and Present Challenges

•Vertical domain vocabulary for DHS and DOJ

•One dozen domain dictionaries

•Technology limited - built using XSD schema

•Core components highly contextual to DHS/DOJ

•Surprisingly – no actual logical models of information!

•Gap with semantic technologies integration

•Limited data content rules and code lists

•Mapping automation support missing

•Multi-year development life cycles


SOA View - Information Sharing Stack and Rules

Privacy and Policy Automation

Identity and Access

Management

Routing / Process Flow

Standardized Metadata

Common Vocabulary

Validation Rules

Business Process,

Enrichment, Routing Rules

Identity, Access Control,

Classifications Rules

Privacy and Policy

Automation Rules

NIEM: rules important; not just data


NIEM is still improving…

•Collections of complex XML Schema•Verbose components•Embedded context in names•Currently facing significant scaling challenges

– Inconsistencies; too much manual management; slow lifecycles

•Dictionary technology incubating•Enhanced code lists mechanism incubating•UML profile is evolving initiative with OMG

– Attempt to marry modelling techniques and XSD Schema syntax

•Policy and Security mechanisms incubating•Need to embrace enabling tools for rapid expansion of

domains and community collaboration


The 8 “D”s and NIEM

•Design•Develop•Deploy•Document•Dictionaries•Discovery•Differentiate•Diagnose

Repeatable, Reusable Process(Exchange Specification Lifecycle)

NIEM IEPD Process

*IEPD - Information Exchange Package Documentation


Integrating into your existing processes and control

•Critical to manage and steer use of NIEM•Ensure business outcomes match requirements•Lessons learned with NIEM

– The earlier in the software development process that NIEM is introduced the better the overall outcomes

– Each step has deliverables – business plan, delivery architecture, data model, alignment reports, schema, test results that can be monitored in your SDLC*

– Monitoring products reinforces reuse as best practice– Tools essential in review processes

•Collaboration tools can coordinate artifacts across projects and participants

•Test bed tools and open APIs reduce integration alignment costs *SDLC – Software Development Life Cycle

Dictionaries: Key Pillar of Information Sharing

Canonical XML Components DictionaryCanonical XML Components Dictionary11

Component Associations and CouplingsComponent Associations and Couplings22

Exchange Templates and RulesExchange Templates and Rules33

W3C Schema and Model RepresentationsW3C Schema and Model Representations44

Delivery Control, Messaging, SecurityDelivery Control, Messaging, Security55

Presentation

Collaboration S

ervices

Artefact relationships

Implementation Artifacts and ExamplesImplementation Artifacts and Examples66

Navigation and Query

Tools

Domains Reuse Library

Components

Automatic Rendering Tools

Deployment Environments and Middleware

XML

User Inserts

Relationship Lookups

TemplateCatalog

Testing Workbench, Rules Engine, Data Samples, Integration ETL

SchemaXML

Models Components

Canonical Dictionary Collections

33


Developing domain dictionaries (EIEM)

•Allows domains to manage their components libraries•Provides consistency for project development teams

– Sets of NIEM consistent XML exchange components– Aligned to enterprise data stores– Optimized for reuse and interoperability

•Save time and effort across the enterprise– Perennial question for developers – when should I use NIEM

components, and when our own local ones?– Are there components already available for that purpose?

•Provide formal mechanisms and procedures to share components and collaborate across SDLC process

•Provide external parties consistent data views

*EIEM – Enterprise Information Exchange Model

EIEM/BIEC

• Business Information Exchange Components (BIEC)– NIEM-conforming XML schema data

component definition that meets a particular recurring business requirement for an enterprise

• Enterprise Information Exchange Model (EIEM)– NIEM-conforming schemas that

define data components to be reused in IEPDs developed by an enterprise; collection of enterprise BIECs organized into a subset and one or more extension schemas


Available Dictionary Tools

•CAM toolkit provides a rich set of tools to create and manage dictionaries– Bottom up harvesting of existing information assets– Top down modelling and engineering– Reuse scoring and comparison reporting

•Dictionary aligned with UN/CEFACT CCTS work– Supports concepts and model– Component renamer supports NIEM Naming and Design Rules

•Practical solution - supports desktop tools– Support for Excel spreadsheet importing and exporting– Can generate UML models– Works with simple Mindmap rendering

•Dictionary Collections– Create collaborative shared sets of dictionaries


Dictionary Management / Acquisition


Value Proposition

•Allow business data analysts to focus on information needs and build data exchanges

•Aligns information with industry standards and enterprise information stores

•Ensures consistent usage and definitions using profile of syntax-neutral terms and constructs

•Components can be derived from existing domain schema and data structures

•Supports providing sharing and collaboration services

•Allows development of supporting analytics tools

The Vision

NIEM Components in

Neutral Dictionary

Representation

UML Technologies

XML Technologies

W3C Schema Tools

Semantic Representations

Rules Technologies

HTML 5 rendering tools

Security solutions

Middleware solutions

39

NIEM DEVELOPMENT

NIEM Concepts, Tools and Adoption

Information Exchange Life Cycle (IEPD)


National Information Exchange Model (NIEM)Tools for enabling interoperability 1

Provides the tools for enabling interoperability at the data layer within and across systems supporting information sharing, while preserving investments in current technology and optimizing new technology development.

COMPLEXITYmultiple

disconnected components + closed coded


CAM Toolkit (http://www.cameditor.org)

• Minimal learning curve for practitioners• Provide technology neutral methods, tools and techniques• Leverage visual metaphors – WYSIWYG• Business Rule Validation• Conformance Test Suites• SQL data mapping / XML generation (code-free methods)

• Creates technical artifacts needed for a NIEM IEPD• Deliver the NIEM domain and core component sets in format that

can be quickly adapted and reused

SIMPLICITYintegrated

components + visual metaphors +

open source

CAMeditor.ORG Project StatisticsSNAPSHOT OF

PROJECT ACTIVITIES

130,000+ CAMeditor.org page

visits to site

165+ countries have downloaded tools; 30% of visitors are

from U.S.;700+ downloads

weekly

2000+ student views of online video training

resources

8 languages now available

www.cameditor.orgwww.niemtrainingvideos.org 43

GOVERNMENT OPPORTUNITIES

Summary


Key Technology Needs

• Delivering on the Open Data Digital Strategy Vision for government

• Open Standards and Open Source based

• Plug and play with code-free templates and rapid development

• Supports NIEM and Open XML exchanges

• Leverages deployed technology today

• Workflow integration

• Dictionary component management

• Secure messaging delivery and partner management

• Delivered across-platform and device

• End-to-end security & governance


Opportunities / Challenges

•Open Data movement – delivering on promise of

government transparency

•Enabling next generation digital economy and

information sharing

•Providing improvements and efficiencies for intra-

government information sharing

• Integration into SOA delivery stack

• International collaboration on information standards


TECHNOLOGY RESOURCES

Addendum


Payload XML/JSON

Data Exchange

Open XDX - Conceptual Overview

Existing Information Databases

SQL

Open Data XML

Open-XDX

SQL

Rapid

Deploy

JDBC connection

Control template contains information of the exchange structure design and DB mapping rules of data tables and columns

Template

Structure

Rules

DB Mappings

ParametersConfiguration

Packaging

Delivery Service

APISend

RESTful Webservice (WADL) or

SOAP (WSDL)


NIEM IEPD Reports

DictionariesDiscovery

NIEM IEPD / Exchange Delivery Lifecycle

Diagnose

Differentiate

Design

Develop

Validated

Templates /

Schema

XSD Schema

XML Samples

XMI / UML Models

Production

Results

Requirements

Updates

Documentation

Drag and Drop Visual

Designer

DocumentDeploy

Exchange

Templates 1

2

3

4

5

Required IEPD artifacts


IEPD Delivery Tasks / Tools Matrix

CAM CAMVOpen-XDX

(OPX)Exchange Schema Development X

Canonical Model Development / Generation X

Exchange Schema Generation X

NIEM Compliance Checking / Reporting X

WSDL / JAXB Binding Generation X

Operation Message Test Case Generation X

Production Services Message Generation X

SQL Schema Mapping X X

NIEM IEPD Artifact Generation X

- Cross-reference spreadsheet X

- UML model X

- Business Rules Report X

- XSD schema package X

Conformance Test Suite X

Run time XML instance validation X

In order to be NIEM-conformant, the IEPD must adhere to:

1. NIEM Conformance Document

2. NIEM Naming and Design Rules (NDR) v1.3

3. NIEM Model Package Description (MPD) Specification v1.0


XML Technology ComponentsNIEM: need to manage complete aspects of exchange delivery, not just the structure and content data model

Swim Lane View

UML Technologies XML TechnologiesW3C Schema

This is what NIEM exchanges are delivered with on-the-wire!

UMLXMI

XSD Namespace

EnumerationsXSLT

XQueryXPath

XML

Patterns

Associations

CardinalityStereotypes

InheritanceFacets

Elements

Attributes

ContentComplex Types

Audience:

Data Modellers / Designers

Audience:

Software Developers and Middleware Engineers

Audience:

General content delivery and applications; content engineers; rules representation systems; multimedia delivery; geospatial systems

Objects

Web

HTML

JavaScript

JSON

REST

SQLSOAP

Audience:

Web application developers

Open-XDX

53


Dictionaries Semantics Approach

•Abstract dictionary representations with strong

semantics, rules and code lists support

•Schema, models, documentation are generated from

dictionary definitions automatically

•Contextual mechanisms allow linear number of true

core components – not exponential growth

•Easier for practitioners to leverage with more

predictable and repeatable results

•Enable domain use horizontally across government

•ETL mapping automation support

•Continuous collaboration environment

insights for success - ottawa, may 2013 david webber, information architect, oracle public sector -...

Documents

data new business

public data

data narrow

new services open data

real world data

information architect

available information

information purposes