21st Century FrogmanData Driven Solutions for Navy SEALs

FINAL INITIAL

1

Problem: Training injuries

lead to BUD/S inefficiency

and the loss of quality

candidates from the

program

Solution: Predictive

algorithm + dashboard to

display safety risk

Problem: Instructors waste

time recording and entering

data during training

Solution: App that

streamlines real-time data

entry

The TeamAnd our fantastic Sponsor and Mentors

Andrew SparksBS Aero ‘03

MBA ‘18Subject Matter Expert

SupportCommander DeCoster

Lieutenant Prybella

Sponsor

Paul Dawes

Mentor

Lisa Wallace

Industry Mentor

Colonel Burns

Military Mentor2

Priyanka SekharBS CS ‘17MS CS ‘18

Machine Learning

Kelsey SchroederBS MCS ‘17

MS MS&E ‘17Strategy

Gerardo RendónBS SYMSYS ‘17

MS MS&E ‘18Design / Data Mining

Customer Interview Breakdown

3

66 Active Duty Military 31 Military Civilians 28 Industry Experts

125 Total Customer

Interviews

The Emotional Rollercoaster

Highlights

Pre - Coordination Challenges

1 - Spring Break Work

2 - Database Disaster

3 - Problems Everywhere

4 - Trip #1 to NSWCEN

5 - Instructor Surveys

6 - Begin MVP

7 - Product Mission Fit!

8 - Betrayal

9 - Development

10 - Trip #2 to NSWCEN,

deliver prototype to Sponsor 4

Team Emotion Flow Over 10 Weeks

Database and Dataflow Challenges

9 Databases in

use at NSWCEN

5

- NSW employee when asked about

current tools for data analysis

“CTRL-F”

Team assessment: They are CTRL-Fing this entire thing up6

NSWCEN Visit #1

Discovery of Our True Beneficiary

Initial belief

about our

Primary /

Secondary

Beneficiary Leadership Instructor Student Medical N6 KM HP12

1

2

Injury underreporting….viewed as a sign of Strength

Overall, 50% would view the classmate as Tough/Dedicated 1

3

Tough / Dedicated

Stupid

Same

Weak

Tough / Dedicated

Tough / Dedicated

- Instructor when asked if he would

change training due to heightened risk

of injury

“No.”

1

4

Discovery of Our True Beneficiary

Initial belief

about our

Primary /

Secondary

Beneficiary Leadership Instructor Student Medical N6 KM HP12XX

Pivot

Discovery

of the True

Primary /

Secondary

Beneficiary Leadership Instructor Student Medical N6 KM HP12

1

5

Instructors’ Greatest Pain Points

1

6

YES! If I could capture observations while in the field using an

app… would greatly enhance grading and cadre to student

debrief/counseling.

- 1st Phase Instructor

Fact. A high pain point for me, because I need my instructors

commenting on as much as they observed as possible.

The easier that is, the more they capture.

- 1st Phase Officer1

7

MVPs Evolve…...Feature-Specific Learnings

- More refined User Interface

- Ability to select recommended

comments or enter free text

- Ability to search by student or

evolution

- Can select multiple students

- Removed positive / negative

comment feature to reduce clicks

- Added audio record feature 1

8

MVPs Evolve…...Feature-Specific Learnings

1

9

Add comment

Report

injury

Internal Readiness Level

Prototype low-fidelity

Minimum Viable

Product

1

2

3

4

5

6

7

8

9

2

0

Deployment RollercoasterProduct

Mission

Fit!!!

Let’s code!!

Supporter or

Saboteur??

People still use

that??

2

1

We’ll build a

web app!

Deployment roadmap

2

2

Open source and make available

to CLD contract winner

5/19 - Coding Begins

6/2 - NSWCEN Visit #2

Proof of Concept Evaluation

Summer 2017 - Contract

Bids Solicited

Contractors Prep Bids

Fall 2017 - Contract

Awarded

-Reduced data entry burden: Less time spent entering data during/post evolution

-Ease of use: User friendly U/I, intuitive note-taking process

-More optimal utilization of instructor expertise:Instructors spend more time training students and less time entering data, which.

21st Century Frogman: Mission Model Canvas

- UI/UX Design- Backend database design- Integration with current systems and standards- Earn customer buy-in- Gather data during alpha release to enable V2 design- Produce technical specs and deliver learnings to leadership

- Continued sponsorship by NSWCEN CSO (CAPT CD), OPS (CDR DD), POC (LT CP)

- NSW N6 Comms/IT

- NSW N3 Database Managers

- CLD Manager

H4D Team:1) Successful U/I proof of concept with placeholder student data2) App provides groundwork for contracted solution

Primary Beneficiary - 1st Phase BUD/S Instructors:1) Reduced data entry time during an evolution by 50%2) Reduced post evolution data entry / synthesizing time by 75%

- Alpha Version of app used by instructors. Valuable feedback is gleaned. - Company that wins new CLD contract uses our app code and integrate into their solution- Push to other military centers with similar needs

Fixed:- Software design & engineering; software products- Travel to NSWCEN (paid for by NSWCEN)

Variable:- Installation and integration; ongoing support

- Software experiencein conjunction with tools

(e.g., Balsamiq, Thunkable)- Credibility from reputation of NSW teammate to aid customer willingness to try product- Guidance and expertise from mentors in industry and military applications

- Need support from NSWCEN (allow implementation) and customer interviews.

- BUD/S instructors who have had previously negative experiences with GideonSoft app

Beneficiaries

Mission AchievementMission Budget/Costs

Buy-In/Support

Deployment

Value

Proposition

Key Activities

Key Resources

Key Partners

- Primary: 1st Phase Instructors - difficult to enter data, stagnant data, student injuries, lost students

- Secondary: NSWCEN / BTC Operations Officer -stagnant data, lost students, student injuries, awareness of “factory floor”

- Other: Students -injuries, instructors

2

3

Mission Achievement

H4D Team:1) Successful U/I proof of concept with placeholder student data2) App provides groundwork for contracted solution

Primary Beneficiary - 1st Phase BUD/S Instructors:1) Reduced data entry time during an evolution by 50%2) Reduced post evolution data entry / synthesizing time by 75%

2

4

Beneficiary Mission Achievement

2

5

20.7

(20 secs per unique entry, 2 minutes to

upload, 30 sec to edit 10% of comments)

hours saved per class

94% time reduction per comment

Increase comment frequency

Generate clean data for analysis (beyond CTRL-F)2

5Higher quality of instruction

9:00 per comment 0:30 per comment

San Francisco Police Academy

250 trainees/year

8 months of training/student

No digital note-taking mechanism

Beyond Naval Special Warfare Center

2

6

Other applications:Military training centers

(Army Special Forces)Amateur/professional sports teams

(Palo Alto High School)

Key Learnings

1) Wet and sandy: not a comfortable combination

2) Customer discovery: essential to problem understanding

3) Sponsor support: goes a long way

4) Security/Legal: can be last-minute blockers

5) Defense: large opportunity for impact

2

7

SPECIAL THANKS TO

Dana DeCoster

Chris Prybella

Jay Hennessy

Christian Dunbar

Scott McCallum

Russell Black

Phil Richey

Brandon Barker

Kevin Mangini

Celeste Machado

Anonymous Brownshirt

Steve Blank

Joe Felter

Pete Newell

Steve Weinstein

Melisa Tokmak

Isaac Matthews

Darren Hau

Jared Dunnmon

Lisa Wallace

Paul Dawes

Curtis Burns

2

8

Appendix

2

9

21st Century Frogman: Value Proposition Canvas

Customer Jobs

Train Navy SEALs

Primary Beneficiary:

1st Phase Instructors

Gains

Pains

Gain Creators

Pain Relievers

- Reduce manual tasks; automate tagging of target activities

Products& Services

Free text data entry app for 1st Phase BUD/S evolutions

- Less time wasted entering redundant data

- More streamlined data-entry experience during training evolutions

- Having to enter all data twice (field and then office)- No way to pre-populate comments on paper

- Reduce data entry burden for instructors through automation

3

0

21st Century Frogman: Value Proposition Canvas

Products& Services

Free text data entry app for 1st Phase BUD/S evolutions

- Reduce data entry burden for instructors through automation

Customer Jobs

Build and deliver high quality, high character

SEALs to the community - Feeling of not knowing

what is going on down onfactory floor

- Attrition of “hard” guys- Sub-optimal utilization of

instructor expertise

Secondary Beneficiary:

NSWCEN / BTC

Operations Officer

Gains

Pains

Gain Creators

Pain Relievers

- Instructors are happy- Instructors spend less time on

redundant tasks (organizational efficiency)

- Instructors spend more time monitoring students, which could improve safety and reduce injuries

3

1

21st Century Frogman: Value Proposition Canvas

Products& Services

Free text data entry app for 1st Phase BUD/S evolutions

- Easier Data entry for instructors

Customer Jobs

Survive Training so they can hunt

bad guys- knees, shoulder, back- Instructor scrutiny

- cold water- sand in crevices- illness, chafing- sleep

deprivation

Other Beneficiary:

Students

Gains

Pains

Gain Creators

Pain Relievers

- Happier Instructors

TBA

3

2

Instructors (BUDS 1st phase) day in the life

3

3

6:00 am - 6:30 am: Get breakfast

6:30 am - 7:00 am: Brief students on what evolution will consist of

7:00 am – 12:00 pm: Take out the boats and do some runs in the ocean with students. Come back, have students do a PT session, time them and take notes on their performance. Then teach SEAL skills class.

12:00 pm – 12:45 pm: Lunch

1:00 pm – 3:00 pm: Timed evolution for students; take notes on performance.

3:00 pm – 3:30 pm: Input student performance data on spreadsheets

3:30 pm – 5:00 pm: Oversee student pool exercises

5:30 pm: Finish inputting last minute notes, make final arrangements, leave work

Stay until 8:30 for dinner and another evolution (about two or three times a week).

Customer

Archetype Leadership Instructor Student Medical N6 KM HP

Position CDRE / XO

OPS / CMC

First phase

BUD/S

BUD/S

Candidate

Corpsman,

MD

Comms, IT Knowledge

Management

Contracted

Company

Age - M/F

Rank

36-50 Male

CAPT / CDR /

Master Chief

25-35, Male,

Petty Officer,

Chief

18-30 Male

Seaman,

Petty Officer

25-40 M/F

HM1, HM2,

LT-CDR

20-40 M/F

Petty Officer,

Chief, CIV

25-50 M/F

Contracted

Civilian

30-50 M/F

Civilian

Motivation Safely &

efficiently

produce elite

Navy SEALs

Hold standard

high, safe

training,

supervise

evolutions

Finish BUD/S,

serve country,

be a part of an

elite team

Treat injuries

effectively,

ensure safety

Avoid IT

violations,

keep comms

working

Serve country,

steady 9-5

job, keep boss

happy

Manage

NSWCEN

data, get next

contract

Pains Stagnant Data

High Attrition

Oversight

Inputting data,

unreliable

software

Instructors,

cold water,

sand, injuries

Redundant

data entry,

unreliability

IT violations,

restrictive IT

security

Tasks that are

outside their

scope of work

Competitors,

contracting,

restrictive IT

Gains User friendly

dashboard

User friendly,

automatic

updates

Injury

prevention

Reduced data

entry, less

hurt students

New tech that

is allowed 3

4

User Flow (1st Phase Instructors)

3

5

Record comments

during evolution on

notepad or Notes app

CLD

After evolution, type

comments into Excel

spreadsheet

Email spreadsheet to

CLD contractor (HP)

HP inputs data into

CLD database

Record comments

during evolution on

notepad or Notes app

After evolution, type

comments directly

into CLD

CLD

Wh

at

we

th

ou

gh

tR

ea

lity Integration with CLD is a must

~80% (79.2%) of comments fall in

the top 20% of categories (by frequency of use)

28% of comments fall under

“Other - Negative”

Pareto reigns supreme

3

6

Behavior category # %

Other - Negative 369 28

Demonstrated less than 100 percent effort 133 10

High Performer 115 8.7

Malicious Boat Ducking 99 7.5

Physically weak 94 7.1

Other - Positive 65 4.9

Poor Performance Counseling 64 4.9

Displayed unprofessional behavior 39 3.0

Exemplary performance 32 2.4

Consistently demonstrated leadership 31 2.3

TOTAL 1041 79%

Emerging Vision

Step 1: MVP

Team Frogman Builds

and deploys

Step 2: Integrate with

analytics to provide

attrition stats, medical

drops, real time analytics

on desktop and mobile

Step 3: Wearables for

collecting performance

data in real time. Direct

input to analytics

dashboard.3

7

Emerging vision

3

8Analytics

Admin dataschedules,

rosters

Performance datafree-text comments,

run/swim times,

incident reports

Emerging vision

3

9

V1 V2 V3

Admin data Daily training schedule Manual Manual Auto

Phase roster Manual Auto Auto

Performance

data

Free-text comments Auto Auto Auto

Incident/injury reports Auto Auto Auto

Run/swim times Manual Manual Auto

Analytics Comment analysis No Yes Yes

Attrition statistics No Yes Yes

Injury pinpointing No Yes Yes

4

0

Emerging

Vision STEP 1 STEP 2 STEP 3

Feature

Summary

GET: Free-text comments,

formatted to compatible

spreadsheet, roster

manually inputted/edited

KEEP: prefilled comments,

prioritization, analytics

GROW: Attrition stats. 1st

Phase current stats

dashboard, roster auto-

populated

2b: Comment analytics

(identify top guys, NLP

characterization)

3a: Graph of performance

vs. character, fine-tuned

stats, real-time injuries,

student geo-tracking

3b: centralized schedule,

preloaded into app

Instructor

Value Prop

-Less time spent entering

data during/post evolution

-User friendly U/I

-Automatic roster updates

-Quick reference student

performance trends

-Attendance/times recorded

automatically and instantly

-No more “lost” students

Leadership

Value Prop

-Instructors spend more

time training students and

less time entering data,

which safety & injuries

(hypothesis)

-Trends, comparison to

historical averages

-”Factory floor” awareness

dashboard, “red flag” alerts

-Analytics and awareness

-Be able to see 2x2 matrix

graph of performance and

character real-time

Student database integration SMS integration, $ (RFID)

UI-specific learnings• Selection of multiple students• Suggested comments (positive and

negative)• Injury reporting option• Search bar for students in addition to list• Alphabetical ordering instead of ordering

by groups/boat crews• Select evolution first• Big buttons with text• Voice feature (not shown in current

mockup)4

1

Current mockupTest here: https://h4d.bubbleapps.io/version-test

4

2

Time saved calculation

20 seconds/per comment, but 0 seconds per duplicate(20 seconds) * (52% unique comments) = 10.4 seconds

30 seconds to edit a comment, done to about 10%(30 seconds) * (10% of comments) = 3 seconds

2 minutes to upload, will upload ~10 comments at a time(2 minutes) / (10 comments) = 12 seconds

Time spent per comment: 25.4 seconds (round to 30)Time saved per comment: 9:00 - 0:30 = 8:30

1315 comments/9 classes, so assume 146 comments/classTime saved per class = (8.5 minutes) * (146 comments) = 20.7 hours

4

3

4

4

4

5

4

6

4

7

4

8

4

9