1

Military Operations Research Society (MORS) Cyber Analysis WorkshopOnline Plenary Session

21 October 2008

Whitney, Bradley, & Brown (WBB) Consulting, Reston, Virginia, 28-30 OctoberGovernment Senior Leader virtual review, 30 October

Registration Information: MORS Office (703) 933-9070 or www.MORS.org

2

Defense Connect Online (DCO)

• Connections for this meeting– DCO https://connect.dco.dod.mil/cyberanalysis

with audio through computer speakers– Audio backup, call 877-206-5884 with code

547836 for teleconference

• Anyone not connected?– If no audio, respond in chat pod– If no visual, speak up or call teleconference

3

DCO Rules of Engagement (ROE)

• Many individuals online today– We want all of your inputs!– Most of you can not talk

• DCO has the communication capabilities—this is our ROE– Chat: any time to everyone or an individual– Comment/Question pod: write any time, addressed at planned

periods– Suggestion pod: write any time, not reviewed in this session– Attendee/status: indicate “have a question” to interrupt

• Other DCO capabilities– Agenda pod– Polls will occur later in this session

Agenda

• STRATCOM/J5 Address • AF/A9 Video• Review Workshop Approach

– Start with M&S Requirements– Tracks develop challenges and recommendations– Discipline Groups improve and add to recommendations– Senior Leaders review and guide

• Summary of M&S Requirements• Tracks Plans• Discipline Groups• Solicit intended participation• MORS Opportunities

5

Work Shop Chair’s Welcome

Dr. Mark A. GallagherSecretary, MORSDeputy Director for Resource Analysis, HQ USAF/A9R

• Thanks!!!– Dr. Henningsen, Headquarters Air Force

A9, our official workshop co-sponsor– Ms. Susan Shekmar, OSD NII, our official

workshop co-sponsor– Mr. Moore, President of WBB Consulting

for hosting our meeting 28-30 October at your wonderful facilities

– Mr. Cares, Alidade Incorporated for providing SharePoint site for planning

• New initiatives– Conducting both unclassified and

classified tracks so uncleared individuals can contribute to solving these national security changes

– Using wikis and these online sessions so you can guide the agenda and discussions

– Vetting recommendations to senior government leaders during workshop

• Request your active participation!• Questions?

6

Thoughts from USSTRATCOM

• Ability to operate, defend and fight in and through cyberspace is analogous to where Air Power was during the interwar period

• The cyber domain simultaneously intersects every other domain

• Understanding the cyber threat and improving our analytic approaches and techniques for cyberspace are key challenges

• I look forward to hearing the result of the workshop.

Mr. Michael Elliot, SES, Deputy Director, Plans and Policy (J5A), U.S. Strategic Command

7

Video from our Workshop Co-Sponsor• Her views on the needs for

cyber analysis in this video• Analytical techniques with

capabilities similar to operations and acquisitions in other areas

• Cyber offense is more difficult than the most challenging kinetic actions, combating terrorists

• Cyber defense is more challenging than preventing crime

• Cyber is crucial to our national securityDr. Jacqueline R. Henningsen, SES

Director for Studies and Analyses, Assessments and Lessons Learned, Headquarters U.S. Air Force

MORS Sponsor and FellowCyber Analysis Workshop Co-Sponsor

8

Workshop Goals and Objectives• Goal: Advance the analytical foundation for cyber actions

for national security• Objectives

– Understand the cyber threat– Improve analytical approaches that support cyberspace

operations• Address cyber analysis including modeling & simulation

requirements• Critique present and proposed analytical approaches and

techniques• Prepare recommendations to improve cyber analysis

– Out brief senior government leaders on recommendations– Write workshop report with recommendations and justifications

Workshop Leadership• Staff Functions

– Security and Facilities (Greg Ehlers)– Virtual Collaboration (Scott Hamilton, Todd Hamill)– Physical Meeting (Jeff Cares)– Taxonomy (Bob Koury)– WBB Site Coordinators (Dennis Baer and Tim Hope)– Senior Leader Coordination (Greg Keethler)– Cyber Modeling and Simulation Requirements (Chris Jeffrey)– Workshop Bulldog (Mark Reid)

• Matrix participation between tracks and discipline groups– Tracks – desire co-leads external to DoD for unclassified tracks

1) Cyber Environment (Greg Larsen)2) Cyber networking for situation awareness and C2 (Len Popyack and Pat Allen)3) Cyber vulnerabilities, protection, defense (Bud Whiteman)4) Cyber deterrence (Pat McKenna, Terry Pudas)5) Cyber exploitation and offensive operations (Bob Morris, Jim Pickle, Linda Namikas)6) DoD Web-Policy Impacts on Cyber Operations (Dennis Murphy, Jason Dechant)

– Discipline Groups1) Optimization (Lee LehmKuhl)2) Decision Analysis (Hunter Marks, Rafael Matos)3) Simulation (Sandy Thompson, Laura Nolan)4) Computer Science (Jarret Rush)5) Social Sciences (Deanna Caputo)

• Keynote Speaker: Dr. Ronald C. Jost, Deputy Assistant Secretary of Defense

10

Workshop Organization

• Tracks have lead on addressing modeling and simulation requirements• Discipline (academic specialty) groups suggest approaches to track challenges• Physical meeting is primary track sessions with discipline group at end of day• The workshop is “working” in participants will develop approaches to meet the

three sets of Cyber Modeling and Simulation requirements

Virtual Collaboration (Hamilton, Hamill)Tue Wed Thur

Cyber Environment (Larsen)

Cyber C2 (Papyack, Allen)

Cyber Defense (Whiteman)

Cyber Deterrence (McKenna, Pudas)

Cyber Offense (Pickle, Namikas, Morris)

DoD Web-Policy Impacts (Murphy, Dechant)

Optimization (LehmKuhl)

Decision Analysis (Marks, Matos)

Simulation (Thompson, Nolan)

Computer Science (Rush)

Social Sciences (Caputo)Multi-level Security (Ehlers)

Physical Meeting (Cares)Aug, Sep, & Oct

Tra

ck

s

Ple

nary

Ses

sion

(G

alla

ghe

r)

Ple

nary

Ses

sion

(G

alla

ghe

r)

Ple

nary

Ses

sion

(G

alla

ghe

r)

Gov

t Sen

ior

Lead

er R

evie

w (

Ke

eth

ler)

Dis

cip

line

Gro

up

s

11

Physical Meeting ScheduleTime Tuesday

28 Oct

Wednesday

29 Oct

Thursday

30 Oct

0830-1000 Plenary Session

Keynote: Dr. Ron Jost, DASD for C3, Space and Spectrum

Tracks Discipline Groups

1030-1200 Tracks Tracks Tracks

1200-1300 Lunch Lunch Lunch

1300-1430 Tracks Tracks Government Senior Leader Defense Connect Online (DCO) session to review workshop recommendations

1500-1630 Discipline Groups Discipline Groups

Evening Social Wrap-Up Wrap-Up

9 Ninety-Minute Sessions: 6 for Tracks and 3 for Discipline Groups

Track and Group Interface• Tracks must write summary for end-of-day

– Any agreed challenges, recommendations, and actions– Issues to be addressed– Specify classification with unclassified version, if possible

• Discipline Groups– Review tracks summaries– Write specific recommendations with justification– May develop own challenges, recommendations, and actions– Specify classification with unclassified version, if possible

• Attendees participate in both a track and a discipline group– Track is in-depth focus within an area– Discipline group provides overview and a different perspective of the

challenges12

Recommendation Format• Challenge: Express current limitation or problem that analysis can address• Recommendation:

– Describe actions to implement recommendation (samples types below)• Need for organization cooperation between …• Need funding for …• Improve testing by …

– Characterize each recommendation• Priority (critical, important, needed, enhancing)• Urgency (immediate < 1 yr, near-term 1-3 yrs, long-term >4 yrs)• Resources (inexpensive < $1M, medium cost $1M-$10M, expensive > $10M)

• Senior leaders will assess– Priority (critical, important, needed, enhancing, no value)– Urgency (immediate, near-term, long-term, not needed)– Feasible (likely, probably, challenging, not possible)

13

Planning Tool Improvement• Challenge: Need improved planning tools for cyber

operations• Recommendation: require planning estimates

– Require effectiveness estimates with indication of technique accreditation status

• All approval packages

• Cyber tests and experiments

– If planning technique is not accredited, capability provider must submit it to IO JMEM for review

– STRATCOM lead annual review of accredited planning models and report to OSD OT&E and NII

• Characterization (Important, Long-Term, Challenging)

14Notional Recommendation Only

Cyber Tools Classification

• Challenge: Many cyber tools may be over classified as SAR/SAP

• Recommendation: Develop and implement a risk assessment decision aide to guide tool classifications– Commission team of analytical organizations to propose approaches and

develop prototypes– Arrange independent analytical review of proposals, document strengths

and weakness, and recommend classification decision aide– Mandate application of decision aide in classifying tools

• Characterization (Important, Long-Term, Challenging)

15Notional Recommendation Only

16

Workshop Report

• Workshop will produce a worthwhile written report– Makes current analysts aware of other initiatives– Brings new analysts up to current capability– Provides recommendations to senior leaders on how to proceed

• Report Content– Summarizes of background – Identifies issues– Assesses current analysis approaches– Evaluates enhancements or alternative approaches– Recommends steps to develop or implement improved analytical

approaches• Tracks and Discipline Groups need to write their good ideas!

Cyber M&S Requirements Sources

• ASD(NII) “determine the M&S requirements for EBO in cyberspace”– 72 requirements (broad analytical tasks)

• IO JMEM COCOM inputs– 20 requirements (more tactical requirements)

• Air Force Agency for Modeling and Simulation (AFAMS) – 5 organizations brainstorming thoughts

Cyber M&S Requirements

18

Track Unclassified Classified Total

All Tracks 11 1 12

Environment 0 0 0

C2 5 3 8

Defense 17 18 35

Deterrence 2 1 3

Offense 6 28 34

Web-Policy 0 0 0

TOTAL 41 51 92

These requirements are mostly general analysis tasks.

19

Unclassified and Classified Tracks

• Cyber Environment – Dr. Greg Larsen, IDA

• Cyber Situational Awareness and Command and Control – Dr. Len Popyack, AINFOSEC, and Dr. Pat Allen, JHU/APL

• Cyber Vulnerabilities, Protection, and Defense– Bud Whiteman, BAH, USSTRATCOM & IO JMEM

• Cyber Deterrence– Pat McKenna, USSTRATCOM, and Terry Pudas, NDU

• Cyber Exploitation and Offensive Operations – Col Jim Pickle, HQ AF GCIC; Col Bob Morris, 67 NWG/CC; Linda

Namikas, ACC 346 Test Squadron

• DoD Web-Policy Impacts on Cyber Operations– Prof. Dennis Murphy, Army War College; Jason Dechant, IDA

Classified sessions will be limited to Secret No Forn

Cyber Environment Track• Lead: Dr. Greg Larsen, Institute for Defense Analyses (IDA)• Track classification will be Unclassified• Cyberspace is the emerging center of gravity for global interactions• Critical issues have many implications and include:

– The space is “constructed” not natural– The effective use of cyber capabilities depends on agile adaptation to changes in the

environment– The increasingly strong dependence of other capabilities operating in other

environments on the cyber environment complicates the M&S challenges– Cyber warfare cannot and should not be equated to information warfare or computer

networks warfare– Cyber M&S must incorporate human behavior into operations in, through, and from

cyberspace

• This track is focused on this wide array of issues and frameworks that determine the credibility, relevance and significance of cyber analyses.

• Questions?

21

Cyber Situational Awareness (SA) and Command and Control (C2)Track

• Leads Dr. Len Popyack, AINFOSEC, and Dr. Pat Allen, JHU/APL• Track classification is Unclassified• Purpose: Identify issues and recommend actions for analysis of cyber

support to C2 and SA• Topics:

– Broad issues• Scalability & Applicability, Analysis of Cyber Support

– Domains • Allegiances and Sides, Instruments of National Power, Timeframes

– Technical Topics• Connectivity, Content & Measures, Security, Visualization, Tools

– Other topics not listed above

• Questions?

22

Cyber Vulnerabilities, Protection, and Defense Track

• Lead Bud Whiteman, BAH at USSTRATCOM, IO JMEM• Track classification is SECRET/No Foreign Nationals• Our nation, including forces contributing to national

security, rely on cyber systems and services  – What are the vulnerabilities of these systems?  – How do we protect and defend them?  

• This track focuses on analytical methods to address these questions– Describe the capabilities of current tools– Determine what is need to meet the requirements

• Questions?

23

Cyber Deterrence Track• Leads Pat McKenna, USSTRATCOM, and Terry Pudas, NDU• Session classification is Unclassified• Track topics

– How is deterring cyber similar/different from “traditional” deterrence?• Who is the actor (e.g., state, non-state, individual)?• Attribution vs. non-attribution vs. not attributable• Lack of precedents, red lines, and established declaratory policy

– What analytic capabilities are required?• Across academic disciplines (Social sciences, OR, etc.)

– What analytic tools exist? What are the analytic gaps?– War gaming deterring cyber issues

• Is it a valuable approach?• What has been done in the past?• What are the “best practices”?

– How do you assess actions to deter cyber?• What is the contribution of cyber defense to deterring cyber?• How are 2nd (nth) order implications represented?

• Questions?

24

Cyber Exploitation and Offensive Operations Track

• Leads Col Jim Pickle, HQ AF GCIC, Col Bob Morris, 67 NWG/CC, and Linda Namikas, ACC 346 Test Squadron

• Session classification is SECRET/No Foreign Nationals• Big Questions:

– How can the US use cyber capabilities?– How can we plan and assess the effectiveness of these techniques?

• Focus questions:– How is cyber similar/different from “traditional” exploitation/offensive actions?– What analytic and M&S capabilities are required? What analytic/M&S tools exist? – War gaming cyber conflict: How ? Is it valuable? What are the “best practices”?– How do you assess cyber offensive actions? What are meaningful metrics?– What are appropriate Cyber CONOPs?

• Planned approach:– Overview of real-world ops– CNA JMEM Successes (TVM/WCM and Models)– M&S support needs from community– CONOP Development process– Metrics to support COCOMs and OPLANS

• Questions?

25

DoD Web-Policy Impacts on Cyber Operations Track

• Leads Professor Dennis Murphy, Army War College; Jason Dechant, IDA

• Track classification will be Unclassified• Current and future war consider battle of ideas on par with battle of

arms• Internet is crucial

– Routine business and communication – Message delivery in strategic communication

• Defend or Attack in the Cyberspace?– Defending the network for our use– Use the network offensively to get out our message proactively

• Managing risk and achieving balance– Current policy applies centralized control and execution to protect the networks– Decentralized execution allows for proactive and reactive speed to send the

message• This track is focusing on analysis approaches that can help the

government implement balanced policies in support of cyberpower.• Questions?

26

Unclassified and Classified Discipline Groups

• Optimization– Dr. Lee Lehmkuhl, MITRE

• Decision Analysis– Hunter Marks, USSTRATCOM– Rafael Matos, WBB

• Simulation– Dr. Sandy Thompson, PNNL– Laura Nolan, JHU/APL

• Computer Science– Jarret Rush, MITRE,

• Social Sciences– Dr. Deanna Caputo

Classified sessions will be limited to Secret No Forn

Optimization Discipline Group• Focus on identifying contributions of optimization techniques to

determine best Courses of Actions (COAs), potential vulnerabilities effect points, and resource tradeoffs arising across all tracks. 

• Questions:

– What optimization techniques can provide insights?

– How do we address the softer qualitative aspects?

– How can we minimize limitations of optimization approaches?

– What are the assumptions of approaches and the effects when those assumptions are violated?

– How can post optimality analysis be used most effectively?

• Discipline Group Lead: Dr. Lee Lehmkuhl

One Group: Unclassified

Decision Analysis Discipline Group

• Focus on identifying contributions of decision analysis to the analytical challenges arising across all tracks. 

• Techniques:

– Value-Focused Thinking

– Decision Trees

– Influence and Affinity Diagrams

• Issues

– Determine decision-maker and approach weights and ranks

– When have conditions change sufficient to modify model weight?

• Discipline Group Leads:– Unclassified: Rafael Matos, WBB– Classified: Hunter Marks, USSTRATCOM

2 Parallel Groups: Unclassified and SECRET/No Foreign Nationals

Simulation Discipline Group

• Focus on identifying needs and contributions of simulation to the analytical challenges arising across all tracks. 

• Questions:

– What systems should be simulated?

– What simulation research is required?

– What groups (users) require simulations?

– What types of simulation tools exist and what are good qualities?

• Discipline Group Leads:– Dr. Sandy Thompson, PNNL

– Laura Nolan, JHU/APL

One Group: Unclassified

Computer Science Discipline Group

• Focus on computer technologies affect our ability to conduct cyber operations in the areas for each track. 

• Questions:

– How are the technologies affecting our ability to analyze cyber operations?

– Are the analytical approaches addressing the critical aspects of the technologies?

• Discipline Group Lead: Jarret Rush, MITRE, supporting AFRL/XPC

One Group: Unclassified

Social Science Discipline Group• Focus on social and human dimensions that affect our ability to

conduct cyber operations across each of the track areas. 

• Questions:

– How do we address human impacts on effectiveness of cyber operations? Are the track approaches addressing or ignoring critical aspects?

– Are the threats of hackers, terrorists, non-state actors, and states being adequately addressed?

– How can behavioral influence be modeled for operational purposes vs. predictive vs. descriptive purposes – what is “good enough” in which situations?

– How can we apply the findings and methodologies of research done in the social sciences (e.g., psychology, anthropology, sociology, behavioral economics, etc) to the cyber problem?

– Discipline Group Lead: Dr. Deanna Caputo, MITRE

One Group: Unclassified

32

Senior Leader Out brief• Senior Leaders to be Briefed Real-time on Workshop

Recommendations• Approximately 10 senior government leaders from the analysis or

cyber communities– Participate in person or via on-line DCO session– Review Recommendations from Tracks and Discipline Groups

• Feedback solicited in four aspects:– Priority of the challenge/recommendation

• Critical, important, needed, enhancing, no value

– Time urgency of implementing recommendation• Immediate, near-term, long-term, not needed

– Feasibility of the recommendation• Likely, probable, challenging, not possible

– Additional insights on the challenge/recommendation• This real-time feedback will be incorporated into the workshop report

– A distinctly new feature of a MORS Workshop

Your senior leaders (SES, Generals, and Admirals) may participate! Contact gregory.keethler@lmco.com or (407) 356-3119

33

WBB Facilities

• The physical meeting on 28-30 Oct – WBB Consulting facilities in Reston, Virginia– Sheraton Hotel is next door

• The facilities are nice and spacious

• Almost all rooms have internet capability– Senior Leader DCO session can be projected

in the various rooms

• Questions?

Workshop Security• Two levels of Classification

– Unclassified with green badges– SECRET/NO FORN with red badges

• Clear participants may transition from one classification to the other

• All participants– Monitor discussions and stop individuals before they

say to much– Check attendance in classified rooms

• Be sure—protect our nation!

34

35

Audience Polling• Your workshop intentions?

– Registered and attending– Planning on attending, however not yet registered– Still considering– Not attending

• For potential attendees, what is your security clearance?– Uncleared– SECRET/NO FORN

• For potential attendees, what is your preferred track?– List of six tracks

• For potential attendees, what is your preferred discipline group? – List of discipline groups by classification

36

Audience Polling (continued)• Did the workshop use of sharepoint site affect planning?

– Very beneficial, minor benefits, no significant impact, adverse impact, I was unaware of it

• The workshop use of an unclassified wiki was?• Very beneficial, minor benefits, no significant impact, adverse impact, limited impact due to policy restrictions,

I was unaware of it

• The workshop use of SECRET wiki was?– Very beneficial, minor benefits, no significant impact, adverse impact, limited impact due to SIPRNET

access, I was unaware of it

• The workshop use of DCO sessions was?– Very beneficial, minor benefits, no significant impact, adverse impact, no opinion

• I expect the workshop use of discipline groups, rather than a synthesis group, to be?

– significant improvement, minor improvement, don’t care, probably, a detriment

• I expect the workshop online feedback from senior leaders to be– significant improvement, minor improvement, don’t care, probably, a detriment

37

Join MORS• MORS has been supporting the Department of Defense (DoD)

for over 40 years– Improving analysis– Networking experts– Enhancing professional development

• MORS is expanding to national and international security– Added Department of Homeland Security as a sponsor– Initiated a dialog with NASA

• View www.mors.org or call (703) 933-9070 for more details on the society, membership, and registration for this workshop

• Workshop Chair: Dr. Mark Gallagher, (703) 588-6949 or Mark.Gallagher@pentagon.af.mil

• Questions?

Wiki Sites

• www.cyberanalysis.pbwiki.com

• http://www.intelink.gov/wiki/MORS_Cyber_and_Networking_Workshop

38