1

Technologies and Processes Panel

Knowledge Management Solutions for Today’s Warfighter

2

PeoplePeopleProcessesProcesses

TechnologyTechnology

Panel Themes

• Show agile technologies and processes to enhance ability to solve “knowledge management” problems– Uncertainty mitigation drives resources– Do not just throw technology at

the problem: balance technology, processes, and people

• Focus on exploitation (active) of content not management (passive) of knowledge– Proactive deterrence is key objective

• Breakdown your complex problem into simple components and address them individually

Exploit the mind in the loop

3

Knowledge Management FamiliesGOAL: Enhance Situational Awareness

Title Functions Performance Metrics

Examples

Data Quality Needs

(Fidelity)

Speed of Response

(Urgency)

Complexity of Logic

(Exceeds One Biological CPU)

Use Storage and Retrieval Low Low Low

Learning Management Systems, resume database, etc

Re-Use Search and Collaboration Medium Medium Medium

Customer Relationship Management, Supply Chain Management, etc

Exploit Consume and Infer High High High

Decision support systems, Progressive Content Exploitation, etc.

4

Progressive Content Exploitation“Data Should Have Velocity Not Position”

• Consume content as it becomes more useful (I.e. aggregated and relevant)– Ability to assimilate large volumes of content

– Content spans data, information, knowledge, and wisdom

• Exploitation is the routing/use of content and inferences from the content– Capture user requirements in the same framework as used for content consumption

CONTENT

DIRECTIVESCUESALERTSCOMMANDS

SITUATIONALAWARENESS

DATA

INFORMATION

KNOWLEDGE

KNOWLEDGE

WISDOM

5

PCE Architecture

Content Ingestion

(of structured content)

Content Extraction

(unstructuredto structured)

Digitization& Translation

PersistentDatabase

(Case Management, Trending, etc.)

Filtering and RankingLink AnalysisTaxonomy DevelopmentClassificationSummarizationIndexingCategorization

Inference Engine

MessageBrokering

CONTENT

Collaboration & Visualization

Content Manipulation

Content Flow Tasking

6

Architectural Components and Solution Sets

ConnectorFramework

ContentIngestion

ContentManipulation

PersistentDatabase

MessageBrokering

ContentVisualization

CollaborationInferenceEngine

LinkAnalysis

Filtering

Dig

i/Tra

nsl

Uns

truc

ture

d

Stru

cutr

ed

Site Profiler

Malta AIDNORA

NeatTools

PIIE

Malta Server from AttensityNORA (Non-Obvious Relationship Analysis) from Systems Research & Development (SRD)AID (Analytic and Information-based Decisions) from Titan CorporationSDS (Seisint Data Supercomputer) from SeisintPIIE (Professional Interactive Integration Environment) from Digital HarborNeatTools from MindTelSite Profiler from Digital Sandbox, Inc.

SDS

7

Performance Outcome Gap

• Decision process is subjective, ad hoc

• Data fused to create better intel/info products

• Most work done after content ingestion

• Databases – waits to be passed along

• A lot of work never gets done that should

• Everybody trying to solve the big problem

• Time between content ingestion and delivery to user is long (or never gets there)

• Manage and distribute information

• Decision process objective, repeatable

• Data consumed to produce customized cues/alerts/directives

• Most work done before content ingestion

• Content streams – moves fast and smoothly

• A lot of work done that might never be used

• Everybody trying to solve a basic problem – architecture facilitates delegation of duties

• Time between content ingestion and delivery to end user is short

• Monitor and exploit streaming data to detect “significant events”

BEFORE AFTER

8

Transition to DSI

• Our first speaker, Bryan Ware, CTO of Digital Sandbox, has a long track record of ambitious and creative software development endeavors. He will be speaking about the core processes of good decisionmaking needed when implementing any decision support or knowledge management system. This presentation provides a great foundation for this panel’s focus on solution processes not just solutions.

9

Transition to Attensity

• Our next panel member, Brent Janorske, represents Attensity. Brent has 18 years of experience in technical software development and sales. He will be sharing with you the focused functionality of Malta Server – an industry-leading content extraction tool.

10

11

Founded in 2000Based on a decade of research in Computational Linguistics and

Information Extraction10 patents pending In-Q-Tel Portfolio companyCustomers include:

CIA NSA DIA JIVA Dept of Homeland Security Whirlpool John Deere

CompanyCompany

Attensity Corporation

12

The ProblemMountains of

Important Data

Unstructured Data Buried In Documents

and Databases

Unstructured Data Buried In Documents

and Databases

NewswiresNewswires

Web TrafficWeb Traffic

EmailEmail

NotesNotes011010101010100101001101

011010101010100101001101

011010101010100101001101

011010101010100101001101

011010101010100101001101

011010101010100101001101

Cables

RequireStructured Data

in HighlyGranular Form

RequireStructured Data

in HighlyGranular Form

Analytical Tools

13

Closing the Gap

RequireStructured Data

in HighlyGranular Form

Analytical ToolsMountains of

Important Data

Newswires

Web Traffic

Email

Notes

Unstructured Data Buried In Documents

and Databases

Unstructured Data Are Turned into

Structured Data Automatically

“Relational Facts” are extracted

– People, Locations and Actions

are Connected

Unstructured Data Are Turned into

Structured Data Automatically

“Relational Facts” are extracted

– People, Locations and Actions

are Connected

011010101010100101001101

011010101010100101001101

011010101010100101001101

011010101010100101001101

011010101010100101001101

011010101010100101001101

14

Linguistic Parsing Demonstration

15

01101010101010010100100101101010101

01101010101010010100100101101010101

01101010101010010100100101101010101

01101010101010010100100101101010101

01101010101010010100100101101010101

01101010101010010100100101101010101

Unstructured Text Becomes “Relational Facts”

Event Type: Terrorism

Sub Type: Bomb Threat

Potential Bomber: Richard Williams

Arms Dealer: John Glover

Material: C4

Place of Purchase: Cairo

Date of Purchase: Oct. 2, 2002

Surveyed Cairo location for24-hour period with no unusualactivity. 8 known suspectspassed thru without incident. Change of shift occurred at 0800 on 10/2/02. Suspected terrorist, Richard Williams (US Embassy bombing in Nov. 2001) entered premises at 0910. Met with John Glover. Williams left premises at 0954, carrying 2 large containers. After his departure, agents determined that Richard Williams had purchased C4. Recommend analysis of potential bomb targets and continued 24-hour surveillance of whereabouts.

16

Automatically Feed Downstream Processing

“Susan Harrissold plastic

explosives toFred Waxman

in Cairo.”

Linguistic Parse

Extract Specific Events / Attributes

Malta Server

TerrorRelationalDatabase

Or - Automatically load into Analysis Tools

17

Text Extraction Architecture

Texts

Knowledge EngineeringWorkbench

(KEWB)

EventDefinitions

Dictionary

MaltaNLP Engine

ExtractionsDatabase

Litigation Event

Attributes:

Plaintiff

Defendant

Jurisdiction

Brent – why?

18

Event Extraction Demonstration

19

Event Extraction Performance

SpeedSpeed Supports automated feeds to tools for link analysis and trending; Produce alerts

RobustnessRobustness Handles abbreviations, Unknown words, Semi-structured data, All Caps Message Traffic, Misspellings, Bad grammar

AccuracyAccuracy Extracts events from free-form text withup to 95% accuracy; Ensure consistent extraction vs. individual interpretations

20

21

Transition to SRD

• While the Malta Server extracts meaningful information from unstructured content our next application aggregates this content even further to ascertain non-obvious relationships amongst a swarm of seemingly unrelated content. George Hargenrader will be representing Systems Research & Development (SRD) and their flagship application NORA – Non-Obvious Relationship Analysis. George has spent his entire career in developing enabling technologies for the exploitation of information for the intelligence community in one manner or another.

22

George HargenraderDirector Intelligence and DOD Programs

Systems Research & DevelopmentLas Vegas, Nevada

SRD Copyright © 2003

Progressive Content Exploitation

“Relationship Awareness from Systems Research & Development”

Portfolio Company

23

The SRD Solution

• Real-time Identity Management Platform– Provides real-time processing & alerts

• Massively Scalable and Extensible– Supports the ability to model “populations”

• Builds an Entity Centric View– Entity Resolution™ finds the real “who”

• Perpetual Analytics™ and Search– Relationships out to 30 degrees of separation

Provides qualified leads for the analyst!

24

The Mind in the Loop

• Situational Awareness is about context– The right information at the right time

• False positives will overwhelm you• False negatives will absolutely kill you

• Context is about relevance to the situation– Based on an Entity Centric View

• Massive consumption of streaming content• Providing a consistent and coherent view

• Relevance is about population modeling– Driven by real-time Perpetual Analytics™

• More Data = More Needles• Provides active management of content

You can’t ask all the smart questions all the time !

25

NORA Capability

Coherent, real-time operational picture Across 100’s or 1,000’s of disparate data sources Full accountability, audit trail, and reconcilable

Detects obvious and non-obvious relationships Between people and organizations All data remains persistent

Generates leads in the form of alerts Suspect relationships found in real-time Real-time publish and subscribe

Required before transactional pattern analysis

Entity Resolution™ determines when two people are really the same

26

Priority #1: Data Integrity

Address Hygiene

Data Warehouse

Name Standardization

Data Quality/Enhancement

Entity Resolution

Load

Human ResourcesSystems

Internal Arrests &

Watch Lists

Customer Systems

Vendor Systems

27

Full Attribution and Temporal Retention

Entity #934256

NAMESSusan MillerSue Miller-ShawSue Shaw

ADDRESSES460 S. Oak Ave.4737 Cimarron Dr.POB 174541101 Princeton #31

NUMBERS(702) 555-2091(916) 342-6721012-34-56785021 1111 2391 1231

RELATIONSSpouse: John ShawRef: Martin ShayRef: Helen BryantRef: Kelly Jones

28

Entity Resolution

Watch List

Juan M TigarDOB: 07/12/76Alice Springs

ID Card

Joan TigarDOB: 07/12/76Wayland(901) 342-9716

29

Entity Resolution

Watch List

Juan M TigarDOB: 07/12/76Alice Springs

ID Card

Joan TigarDOB: 07/12/76Wayland(901) 342-9716

Public Records

Juan Miguel TigarWayland(901) 342-9716Previous: Alice Springs

30

Entity Resolution

Watch List

Juan M TigarDOB: 07/12/76Alice Springs

ID CardJoan TigarDOB: 07/12/76Wayland(901) 342-9716

Public Records

Juan Miguel TigarWayland(901) 342-9716Previous: Alice Springs

31

Entity Resolution

Watch ListJuan M TigarDOB: 07/12/76Alice Springs

ID CardJoan TigarDOB: 07/12/76Wayland(901) 342-9716

Public Records

Juan Miguel TigarWayland(901) 342-9716Previous: Alice Springs

32

Rules of Engagement

Cannot first mandate upgrades to all source systems– Must not disrupt front line systems; must take the source data as is

Assume and architect for “n” sources and destinations– Cannot reengineer for each new data source or destination

Should not be designed for periodic warehouse refresh loads– Too much data to continue reloading from scratch every month/quarter

Must be reconcilable to sources and destinations– Must be able to conduct audits to verify and ensure accuracy

Must be perceived as a better than source systems– Must be as timely, at least as accurate, and contain a better collective view … or users

will revert to original practices

No users rummaging (connected) into the warehouse– Users interact with systems/data models conducive to their requirements

33

Transition to Titan

• As George stated so clearly NORA provides leads for analysts to follow-up on. Next on our panel, Brian Crowley of Titan Corporation, will describe an ingenious software application that serves to exploit those leads and other relevant content by inferring and deducing the state of affairs for a given situation. Brian has a legacy in commercial software development but is currently focusing his efforts on several DoD and intelligence community customers in the area of advanced content delivery to the customer.

34

Analytic & Information-based Decisions (AID)

Brian Crowley

The Titan Corporation

703.758.6522

bcrowley@titan.com

35

The Decision Maker Dilemmas

• Repetitive processes

• Data overload

• Not necessarily all required information

Real Goal: Spend Timeon Value-Added Problems

Real Goal: Spend Timeon Value-Added Problems

36

Progressive Content Exploitation“Data Should Have Velocity Not Position”

• Velocity affects decisions– State of influencing

factors change

– Timeliness of notification issues

CONTENT

DIRECTIVESCUESALERTSCOMMANDS

SITUATIONALAWARENESS

DATA

INFORMATION

KNOWLEDGE

KNOWLEDGE

WISDOM

DecisionsDecisions

37

Basic Decision Making

• Start w/a question

• Identify influencing factors

• Determine state of those factors

• Weigh factors against one another

• Decide

WHY: To Implement A Course Of ActionWHY: To Implement A Course Of Action

38

Decision Network

• Start w/a question• Identify influencing

factors• Determine state of

those factors• Weigh factors against

one another• Decide

• Root Question• Influencing Nodes

-.• Data Stream(s)

-.• Conditional

Probabilities• Analyze

Building a Bayesian Belief NetworkBuilding a Bayesian Belief Network

39

Analytic and Information-based Decisions (AID)

Consequence

Recognizable

Risk

Likelihood

Susceptibility

Threat Envt.

Org Intent

Mission Effect

Bayesian Network(abbreviated risk management)

Labeling /Signs

Architecture

Press

-User-configurable decision network

-Bayesian analysis of probability relationships capture response modes

-Inference model requires structured data gathering but will provide the responsivenesss andrepeatability required for a real decision support system

-Values for multiple actions may be aggregated to account for interpendencies

What is my facility’s risk level? HML

40

AID Output

41

Technology and Processes

• AID will produce useful insights when the planning leading up to its use has applied proper decision-making principles– Follow the “Do’s of Decision Making”

• The PCE architecture will not be effective unless proven EA methodologies are applied

42

Do’s of Decision Making

• Control the complexity of the decision

• Be systematic: have a process – plan, collect, conclude, make decision, and analyze results

• Build your network using the power of threes

• Be experiential: verify and validate

Adapted from experiences and references: Jones, Morgan. The Thinker’s Toolkit. Random House, NY, NY, 1998; Russo, J.E., at al. Decision Traps – The Ten Barriers to Brilliant Decision-making and How to Overcome Them. Simon & Schuster, NY, NY, 1989; and Thomke, S.H. Experimentation Matters, Harvard Business School Press, Boston, MA, 2003.

43

Transition to Digital Harbor

• Brian detailed the powerful point solution AID while our next presenter will describe the first of two system solutions to be presented in our panel. Steve Nguyen of Digital Harbor will explain the value-added provided by their Professional Interactive Integration Environment (PIIE) that has been deployed to several intelligence community organizations. The flexibility of the tools selected in this panel is exemplified by Steve’s presentation as he will show how they have integrated AID as a utility within their enterprise solution.

44

Work with applications in a new way. Together.

“The objective of composite applications is to improve corporate performance by improving decision making. 

This is accomplished by drawing on the information already available and adding rich workflow, event, and client interface capability.”

- AMR Research

The composite application company

Company OverviewSteve Nguyen

Director, Business Development(703) 476 -7373

snguyen@dharbor.com

45

The Need: Getting a handle on multi-source information

• Deliver improved context and situational awareness to the warfighter

• Deliver LIVE correlated information and real-time situational awareness to operators

• Deliver a common operating picture to C4/ISR commanders and operational staffs

• Move from stove-piped processes to collaborative operations

• Deliver multi-INT (sensor, electro-optical, humint, sigint, imint, etc) feeds into a single view that can be pushed out to theatre-level operators

• Move from Tool-centric to Service-centric architecture

Satellite image of Iraq weapons facility Video surveillance of

facility

12 hour weatherforecast

Human Intelligence

Predator feedsThreat database

Multiple audio feeds

46

ConnectedApplications

Black Box

Black Box

Black Box

Black Box

Black Box

Black Box

C2 INTEL Logistics

Machine to Machine(Back Office)

CompositeApplications: Net-Centric

Interactive UI for Apps, not Docs

A B C DE F G H

I J K LM N O P

R S T UW X Y Z

A B I R A J S T C K Y Z

C2 INTEL Logistics

Human to Machine(Front Office)

Fuse Services into Seamless Apps

“Often, data in one application acts as a key to understanding the full impact of data in another.” - Gartner Group, 2002

What is Required?

47

Information in the “Grid” Must Provide…Information in the “Grid” Must Provide…Semantics, Context & Correlation Semantics, Context & Correlation Must Promote DiscoveryMust Promote DiscoveryToo much is the “brother” to NO INFOToo much is the “brother” to NO INFO

People in the “Grid” Must…People in the “Grid” Must…Work Collaboratively & ContinuouslyWork Collaboratively & ContinuouslyNot DiscretelyNot Discretely

““Grid” Information Sources are…Grid” Information Sources are…Naturally Distributed & CompartmentalizedNaturally Distributed & CompartmentalizedExposed in ApplicationsExposed in ApplicationsNot enough to fuse content, sources & typesNot enough to fuse content, sources & typesWe must fuse applicationsWe must fuse applications

Development of New Solutions Must…Development of New Solutions Must…Be componentized (SoSE)Be componentized (SoSE)Embrace change, Be Adaptable, Embrace change, Be Adaptable, Push/Stress “Time to Value”Push/Stress “Time to Value”

Network-Centric WarfareValue Proposition for a Net-Centric Approach

48

1. Fuse services from multiple applications

2. Correlate information in context

3. Drill down in Real-Time

4. Ask questions across databases

5. Infer links across systems

A Composite Application for Intelligence Fusion

RFI Map

Equipment Resources

ActorOther Tasks

Other Tasks

49

Operators:Create composite reports consisting of mult-int information and analysisUDOP – User Defined Operating Picture

Systems Integrators:Drag and Drop Components into Live, Composite Apps

Developer:Integrate by graphically linking Data, Processes, Rules, Events

Forward Deployed:Get the battlespace picture with live, contextual information - UDOP

Command Staff: Update Processes on the FlyCommon Operating Picture

Composite Applications Benefit All Stakeholders

IntegrativeEnvironment

50

Demo

51

Transition to MindTel

• The next speaker, Dr. David Warner, is an MD and PhD, with a real passion for helping people. His company, MindTel’s, primary mission is to help severely mentally and physically handicapped people communicate. It is this foundation of enhancing the perception of represented information that he has leveraged to provide a unique contribution to content visualization in concert with traditional decision support and knowledge management systems. While many of is deal with ongoing activities where the “last 100 yards” are so critical. Dace is much more focused on the last millimeter – where perception is born.

52

Transition to Q&A

• Now that the prepared panel presentations have been completed I will open up the panel for questions from the floor…

53

Q&A For Panel Two• DSI

– So what is more important, technologies or processes?

• Attensity– The event extraction model that you described is effective when the analyst knows what they are looking for ahead of time.  What about

situations where the analyst really does not know what to look for or what might be important? NLP has traditionally been hard to implement.  What is it about your approach that is better? What value does event extraction offer above and beyond traditional entity extraction?

• SRD– What other names are there for “entity resolution”? Does NORA require a detailed data taxonomy to perform well?

Can entity resolution be used on “things”?

• AID– This approach seems fairly straightforward – why hasn’t this type of application been developed before? Bayesian

mathematics are fairly complex, how are you overcoming the user apprehension barrier?

• Digital Harbor– What are the most counter intuitive similarities between your commercial clients and your IC customers? Compare

and contrast your tool with Enterprise Application Integration utilities?

• MindTel– How can we best prepare our program managers to “think like a neurocosmologist” without having them revolt – is

there a special chant we should learn? Many of your solutions are enabled by rapid prototyping – what is the driving force behind this process?