Computational and Information Sciences Directorate

Army Research Laboratory (ARL)The U.S. Army’s Corporate Laboratory

Barry A. Bodtbabodt@arl.army.mil(410) 278-6659

Data Mining Combat Simulations: Data Mining Combat Simulations: an Emerging Opportunityan Emerging Opportunity

MotivationMotivationMotivationMotivation

• Simulation and statistical analysis are underutilized in helping the commander’s staff to analyze courses of action.

• Battle results are infinite in scope, yet the outcome of any one battle is defined by a unique set of battlefield interactions.

• Key is to recognizing those interactions through development of more informative performance measures unique to the scenario at hand.

ApproachApproachApproachApproach

Use statistical methods Use statistical methods and combat models to and combat models to create a methodology create a methodology

that identifies non-that identifies non-traditional metrics for traditional metrics for

plan evaluation.plan evaluation.

COACOA

BackgroundBackgroundBackgroundBackground

•Focus on wargameFocus on wargame•Disciplined rulesDisciplined rules•Synchronization matrixSynchronization matrix

Military Decision Making ProcessMilitary Decision Making ProcessMilitary Decision Making ProcessMilitary Decision Making Process

Joint Tactical Operation Center, QatarJoint Tactical Operation Center, QatarJoint Tactical Operation Center, QatarJoint Tactical Operation Center, Qatar

Communicate…Communicate… Smart LogisticsSmart Logistics On-board DiagnosticsOn-board Diagnostics Soldier HealthSoldier Health Sensor informationSensor information … …

Network Centric WarfareNetwork Centric WarfareNetwork Centric WarfareNetwork Centric Warfare

The key to any analysis is the set of measures used to The key to any analysis is the set of measures used to represent the performance and effectiveness of the represent the performance and effectiveness of the alternatives being considered. We are relatively good at alternatives being considered. We are relatively good at measuring the performance of sensors and actors, but measuring the performance of sensors and actors, but less adept at measuring command and control. less adept at measuring command and control. Command and control, to be fully understood, cannot Command and control, to be fully understood, cannot be analyzed in isolation, but only in the context of the be analyzed in isolation, but only in the context of the entire chain of events that close the sensor-to-actor entire chain of events that close the sensor-to-actor loop. To make this even more challenging, we cannot loop. To make this even more challenging, we cannot isolate on one target, or even a set of targets but need isolate on one target, or even a set of targets but need to consider the entire target set. Furthermore, network to consider the entire target set. Furthermore, network centric warfare is not limited to attrition warfare … It is centric warfare is not limited to attrition warfare … It is not sufficient to know how many targets are killed, but not sufficient to know how many targets are killed, but exactly which ones and when…exactly which ones and when…

Ref: Network Centric Warfare, 2002Ref: Network Centric Warfare, 2002

Information Requirements in NCWInformation Requirements in NCWInformation Requirements in NCWInformation Requirements in NCW

Simulation DataSimulation DataSimulation DataSimulation Data

• Scenario development

• OneSAF lay down of forces

• OneSAF modified output

• Data supporting modeling

ScenarioScenarioScenarioScenario

Company Objective

Town

BMP-2

BMP-2

BMP-2

T-80

T-80

T-80T-80

T-72M T-72M

T-72M

T-72M

T-72M

OneSAF Screen DumpOneSAF Screen DumpOneSAF Screen DumpOneSAF Screen Dump

Automated Data CollectionAutomated Data CollectionAutomated Data CollectionAutomated Data Collection

• OneSAF Modifications

OBJECT_ID: 100A31

X = 24396.82 Y = 25828.75 Z = 755.72

Vehicle Authorized Undamaged Catastrophic Firepower Mobility

Damage Damage Damage

M2 1 0 1 0 0

Equip/Supplies: Current Lvl Resupply Lvl Avg Per Veh

25mm HE (M792) 625.00 625.00 625.00

25mm APFSDS-T (M919) 325.00 325.00 325.00

TOW (TOW) 0.00 5.00 0.00

7.62mm MG (M240) 2340.00 2340.00 2340.00

Fuel (Fuel) (gallons) 171.00 174.00 171.00

OneSAF ModificationOneSAF ModificationKiller/Victim ScoreboardKiller/Victim Scoreboard

OneSAF ModificationOneSAF ModificationKiller/Victim ScoreboardKiller/Victim Scoreboard

Time Stamp 1010070890

Vehicle ID 1076

Firer ID 1087

Projectile 1143670848

Firer Position: X = 220217.00 Y = 146765.00 Z = 12.37

Target Position: X = 222454.38 Y = 149117.80 Z = 9.99

Vehicle 1076: Hit with 1 "munition_USSR_Spandrel" (0x442b0840)

Comp DFDAM_EXPOSURE_HULL, angle 19.53 deg Disp 0.889701 ft

Kill Thermometer is: Pk:1.00, Pmf:1.00, Pf:0.90, Pm:0.80 Pn:0.80

RANGE 3246.773576

r = 0.990835 kill_type = MF

• Firer and Target Identity and LocationFirer and Target Identity and Location• Type of AmmoType of Ammo• RangeRange• OutcomeOutcome

1076 100A41 vehicle_US_M1

1087 100A23 vehicle_USSR_BMP2

Data Supporting Classification ModelsData Supporting Classification ModelsData Supporting Classification ModelsData Supporting Classification Models

• 228 OneSAF runs• 3 situational snapshots per

run– 10% blue ammo expended– 25% blue ammo expended– 40% blue ammo expended

• 429 data points per run (143 per stopping time)– Number of K, M/F, F, and M kills– Ammunition levels– Number of hits delivered– Range of hits– Number of side hits delivered– Distance to objective– Number of Blue on objective

Response – mission Response – mission accomplished (success) accomplished (success) if an undamaged platoon if an undamaged platoon occupies objective at occupies objective at battle end (MA)battle end (MA)

-other responses includeother responses include MBT and “Eric” strength MBT and “Eric” strength and forces on objectiveand forces on objective

Data Matrix Data Matrix 228 x 434228 x 434

Company Objective

7435 1

3485 0

1 0 Pred

Obs

Correctly ClassifiedCorrectly ClassifiedLoss: 71%Loss: 71%Win: 68%Win: 68%

Overall: 70%Overall: 70%

Slice 1 ~ 2000mSlice 1 ~ 2000m

Or ~ 5 ½ minutesOr ~ 5 ½ minutes

Company Objective

8425 1

2198 0

1 0 Pred

Obs

Correctly ClassifiedCorrectly ClassifiedLoss: 82%Loss: 82%Win: 77%Win: 77%

Overall: 80%Overall: 80%

Slice 2 ~ 4000mSlice 2 ~ 4000m

Or ~ 10 minutesOr ~ 10 minutes

Company Objective8920 1

14105 0

1 0 Pred

Obs

Correctly ClassifiedCorrectly ClassifiedLoss: 88%Loss: 88%Win: 82%Win: 82%

Overall: 85%Overall: 85%

Slice 3 ~ 5800mSlice 3 ~ 5800m

Or ~ 20 minutesOr ~ 20 minutes

Model PerformanceModel PerformanceModel PerformanceModel Performance

Method ComparisonMethod ComparisonMethod ComparisonMethod Comparison

Stopping Time (min)

Discriminant Analysis

CART Logistic Regression

5 ½ 70% 70% 69%

10 80% 75% 74%

20 85% 82% 85%

Percent Correct ClassificationPercent Correct Classificationby Stopping Time and Methodby Stopping Time and Method

AdvantagesAdvantagesAdvantagesAdvantages

– Support prediction for COA performance evaluation

– Provide models identifying key battle parameters for a given engagement, influencing both COA development and commander’s critical information requirements

– Input to CCIRs– Input to contingency plans– Input to tolerances for synchronization

Implementation ModelsImplementation ModelsImplementation ModelsImplementation Models

Reach backReach back

AdvantagesAdvantages-computational power (ARL 9-computational power (ARL 9thth))-more complex analyses-more complex analyses

DisadvantagesDisadvantages-latency-latency-can’t smell gunpowder-can’t smell gunpowder

DistributedDistributed

AdvantagesAdvantages-cheaper boxes (250 OneSAF-cheaper boxes (250 OneSAFboxes used at Ft. Leavenworth)boxes used at Ft. Leavenworth)-closer to action-closer to action

DisadvantagesDisadvantages-depth of a field analysis-depth of a field analysis-automation required-automation required

Why Aren’t We Already Doing This?Why Aren’t We Already Doing This?Why Aren’t We Already Doing This?Why Aren’t We Already Doing This?

• Computer simulation focus has been mainly strategic or oriented toward acquisition. Tactical application has been limited.

• Simulations did not have high enough fidelity for tactical application.

• Simulations were unstable.

• Computing resources were inadequate.

• Necessary communication of inputs had not been imagined.

• Simulation creators do not always talk to statisticians.

A few reasons …A few reasons …

Improvements Here and On the WayImprovements Here and On the WayImprovements Here and On the WayImprovements Here and On the Way

• Stability

• Power Point force laydown of forces

• MS Word OPORD

•Terrain, weather wizzards

• Composable simulations

• After Action Report data

• Man-in-loop allowed

• Sensor advances

• Communication advances

• Computation speed and cost

Catching On?Catching On?Catching On?Catching On?

20

After Action Review

• Situation awareness during the execution of the exercise and afterwards during exercise playback:

– PVD & 3D Stealth display– Statistical charts, tables– OPORD paragraphs– Task Organizations Summaries– Radio/audio playback (Future)

• Mining of collected data to construct MOPs/MOEs• Automatically build AAR presentations & Take Home Package

using COTS Office Automation

PURPOSE: The OneSAF After Action Review component provides the capability to correlate, roll-up, and analyze simulation outputs and visualize the results of the simulation exercise. The toolset allows the analyst to preplan the AAR prior to exercise execution.

Wei-Yin Loh, Regression Tree Analysis of Battle Simulation DataWei-Yin Loh, Regression Tree Analysis of Battle Simulation Data

David Kim, Robust Modeling Based on L2E Applied to Combat David Kim, Robust Modeling Based on L2E Applied to Combat Simulation DataSimulation Data

Warren Liao, Discovery of Battle States Knowledge from Multi-Warren Liao, Discovery of Battle States Knowledge from Multi-Dimensional Time Series DataDimensional Time Series Data

Next UpNext UpNext UpNext Up