discover new talent pipelines by radically reskilling your ......booz allen’s data science journey...

Discover New Talent Pipelines by Radically Reskilling Your Teams: A Case Study with Booz Allen HamiltonMarch 2018

Meet the team

PAGE 3

Jim Hemgen

Senior Associate, Functional Learning

& Development

Cadence WeberLearning Architect, Functional Learning

& Development

Jay Nappy

Head of Operations, Advanced Skills

Academies

Adi Hanash

Head of Product, Advanced Skills

Academies

SOFTWARE DEVELOPMENT | DATA | DESIGN | PRODUCT MANAGEMENT | DIGITAL MARKETING | CYBERSECURITY

Trends in the marketplace

Companies are spending more on recruitment than

ever before.

Digital transformation is rapidly changing

the job market.

Poaching talent does not increase

overall supply.

There are major opportunities to innovate in talent acquisition and workforce development

1 2 3

RECENT HEADLINES:


General Assembly Advanced Skills Academies

Solving Skills Gaps through Training, not Recruiting

Adding critical, in demand digital skills to your teams in order to make them more effective and

impactful

Upskilling Radical Reskilling

Retraining legacy workers to take on new digital roles crucial to the

success of your organization

Talent Pipeline as a Service (TPaaS)

Creating new pipelines of talent by investing in the training and

onboarding of candidates from a wider, more diverse talent pool


Marketing

Admissions

Training

Outcomes

PAGE 8

Each GA Advanced Skills Academy has 4 critical components that can be managed by GA, in partnership with GA, or handled internally depending on your organizational needs.

Robust marketing to source new pools of qualified talent that fit the profile for the position(s) for which we are sourcing and training.

4 Key Components of an Advanced Skills Academy

GA’s data-driven admissions processes uses a combination of Assessments and screening interviews to vet candidates and assess their readiness for training.

GA’s world-class training leverages our wide network of top practitioners, blended learning models, and assessment-led learning paths.

Our outcomes team or career coaches work with our students throughout the program to develop soft-skills and properly prepare them for any interview process.


Key Milestones for Launching an AcademyTo successfully launch an Advanced Skills Academy, General Assembly conducts a thorough research phase that allows us to determine how to contextualize our material to your organization and tailor our MATO approach to your needs.

Research

Stakeholder Interviews and Assessment Deployment

GA conducts stakeholder interviews in order to

contextualize the content of the courses to your

organization and rolls out assessments to establish

internal benchmarks.Timeline: 2-6 weeks

Marketing and Admissions

Source and Vet Participants

In-depth marketing period designed to identify the “Optimal

Profile” for candidates for training. The admissions process

will use Assessments and Screening to vet candidates.

Timeline: ~10 weeks

Training

Blended Learning Approach

GA’s expert instructional team will train candidates through a

combination of assessments, blended learning paths, and

instructor-led delivery.Timeline: 1 -12 weeks depending

on training outcomes

Outcomes

Internal Placement and Role Transition

Throughout the training process, GA’s support team will also work with candidates on soft skills and prepare them for any interview

process post-course.Timeline: 2-4 weeks


Key Milestones for Launching an AcademyTo successfully launch an Advanced Skills Academy, General Assembly conducts a thorough research phase that allows us to determine how to contextualize our material to your organization and tailor our MATO approach to your needs.

Research

Stakeholder Interviews and Assessment Deployment

GA conducts stakeholder interviews in order to

contextualize the content of the courses to your

organization and rolls out assessments to establish

internal benchmarks.Timeline: 2-6 weeks

Marketing and Admissions

Source and Vet Participants

In-depth marketing period designed to identify the “Optimal

Profile” for candidates for training. The admissions process

will use Assessments and Screening to vet candidates.

Timeline: ~10 weeks

Training

Blended Learning Approach

GA’s expert instructional team will train candidates through a

combination of assessments, blended learning paths, and

instructor-led delivery.Timeline: 1 -12 weeks depending

on training outcomes

Outcomes

Internal Placement and Role Transition

Throughout the training process, GA’s support team will also work with candidates on soft skills and prepare them for any interview

process post-course.Timeline: 2-4 weeks


Measuring ROI of Academy Training

Metrics Tracked ‣Monthly active users (online) ‣# of participants ‣# of graduates ‣Daily/weekly course surveys ‣Net Promoter Score ‣Value for time spent

Participation Assessments Capstone Projects Business Objectives

Metrics Tracked ‣ Industry-benchmarked

assessments (when applicable) ‣ Internal benchmarking with

assessments ‣Pre/Post Assessment

performance

Metrics Tracked ‣Capstone projects scores

based on pre-defined rubric ‣Final presentations and

awards assigned by senior leadership

Metrics Tracked ‣Post-training performance

tracking (dependent on internal systems) ‣KPIs determined prior to

training with GA •E.g., Agile team velocity

pre/post training

PAGE 16SOFTWARE DEVELOPMENT | DATA | DESIGN | PRODUCT MANAGEMENT | DIGITAL MARKETING | CYBERSECURITY PAGE 10SOFTWARE DEVELOPMENT | DATA | DESIGN | PRODUCT MANAGEMENT | DIGITAL MARKETING | CYBERSECURITY

Booz Allen Hamilton Data Science 5K

Booz Allen Hamilton at a Glance


• Founded in 1914

• 100+ offices worldwide

• 25,000+ staff members

• 43% of staff work in analytics roles

• 400+ of Fortune 500 as clients

• 5+ USD billion in revenue

• 15th largest contractor for US gov.


What is Data Science?

Legacy Analytics Data Science

Human/Machines Humans Machines

Type of Analysis Descriptive Predictive

Data source Siloed Data Warehouses

Distributed DataStreams

Tools Commercial, Off-The Shelf Open source tools

Outputs Reports Data Products

Follow-up Repeat Analysis Probe Deeper

Data Science is the art of turning data into actions- The Field Guide to Data Science

Domain Expertise

Provide the understanding of the reality in which a problem space exists

Computer Science

Provides the environment in which data products are

created

Analytical Skills

Operations, research,

and mathematics


Booz Allen’s Data Science Journey

2012

Formalize Data Science practice

at Booz Allen.

Launched the Data Science Bowl and wrote

and released the Field Guide to Data Science

2015 2016

Initiate The Data Science 5K, plan to train

5,000 data scientists.

Explore Data Science made

available to the public

2014

@BoozDatascience

Everyone—from the skilled data scientists to a novice who has yet to write their first lines of code—must infuse Data Science principles into their work to capture the opportunity of today’s technology transformation. We passionately believe that Data Science is core to everything we do at Booz Allen, and that most everyone can employ that tradecraft. With this in mind, we launched the Data Science 5K Challenge, our charge to increase our data science footprint with 5,000 data scientists.

Be the industry leader in Data Science.


Booz Allen’s Data Science 5K Challenge Goal and Mission Statement

Booz Allen’s Metrics to Quantify Program Success


Data Science Criteria Metrics

Project Value, Size ● Billability● Direct Labor Value

Demand vs. Vacancies● Open Reqs● Avg Time to Close Open Reqs● Internal:External Fill Ratio for Closed Reqs

Retention Rates ● Retention Rate, Avg. ● Retention Rate (Firm-wide vs. Analytics group)

Supply vs. Capacity ● Staff Aligned Aligned to Analytics● Staff Aligned to Data Science Role

“Attract, Reskill, Retrain”

Why did Booz Allen Hamilton Partner with General Assembly?


1. Engagement

• Applicant communications• Student communications

2. Assessment

• Assessment library• Demonstration of skill mastery • Post-course evaluation

3. Programming

• Capacity planning• Instructor/student ratios• Learning delivery methods• Course customization

4. Business

• Global scale• Business compatibility• Stakeholder management

Multiple companies submitted proposals, and we asked an internal panel to evaluate each company on the company’s ability to execute on four criteria:

General Assembly Stood Out


1. Engagement

• MyGA.com• Engagement Manager

2. Assessment

• Data Analysis, Level 1• Data Science, Level 1• Company-wide diagnostic

3. Programming

• “Part-time Data Science”• Remote/in-person options• Dataset customizations

4. Business

• Location delivery flexibility• Consulting familiarity• GA’s client list

Building an Academy

Understand Current Capabilities

Key Activities: Partner with Business Leaders & SMEs to

Determine What Capabilities are Needed

Pilot, Refine, and Grow

Key Activities: Develop Workstreams, Pilot Program, Collect

Metrics

The Framework We Used


Develop a Vision for Your Initiative

Key Activity:Define the Goals &

Objectives

Determine the Path Forward

Key Activity: Build vs. Buy


Step 1 - Develop a Vision

Objective

Behavior

Knowledge SkillMindset

Determine Solution

Key Questions:● Is there a skill gap? ● Build vs. Buy?

Partner with Experts

Experts should be: ● Diverse● Dispersed ● Dedicated

Assess Skill Gap

Don’t: ● Rely on a “gut”● Pre-determine a solution

Do: ● Gather hard data ● Utilize third party data

Step 2 - Understand Current Capabilities


Identify internal experts, develop workstreams, and take advantage!

● Model program costs

● Identify funding source(s)

● Analyze impact

● Participant billing policies

● Staff deployment

● Content expertise

● Evaluation measures

● Inform learning roadmap


Step 3 - Determine a Path Forward

Communications Performance Financial

● Comms plans

● Program awareness

● Building buy-in


Step 4 - Pilot, Refine, and Grow

2. Pre-work online (Data Fundamental)1. Assess online (DA1/DS1) 3. Upskill in class (Data Science)


Criteria Description Performance Indicator

Learning

Drastic improvements in skills +50% increase in post-course assessment scores

World class learning experience 51 NPS

Participants recognize value 4.5 VTS

Engagement

Participants thrived with the instructional support provided Qualitative feedback

High conversion rate from interest to engaged >1600 Yammer sign-ups; 930 assessment submissions

High, steady rate of collaboration between students and staff 16,719 Slack messages sent since June

Application

Demonstrate python competency through code challenges Project 1 - 100% students met expectations

Perform exploratory data analysis in Python on a data set Project 2 - 100% students met expectations

Create, train, evaluate model fit for regression/KNN models Project 3 - 100% students met expectations

Create a proposal, find/clean data, model data, present findings Project 4 - 100% students met expectations




Area How should it change? Why should it change?

Delivery Format Remove accelerated offering; offer part-time, 10-week, or part-time, 8-week, only

Qualitative feedback from accelerated courses frequently cited pacing as an issue (too fast)

Staffing Employ full-time instructors that teach multiple cohorts Reduce instructor prep-time required by 50%; allow for more access to instructors and more office hours

Pre-course Experience Provide pre-course “installfest”, virtual office hours for all Pilot pre-course VTS 16% lower than in-course; qualitative

feedback showed students felt unprepared for class

Final Project Introduce final project earlier in class Qualitative feedback indicated students wanted more time to work on final projects

Advanced Classes Offer advanced classes in specific, focused data science topics Engage participants from pilots and analysts too advanced for DS

Fundamentals

End-to-end Experience Be more explicit about pre-, in-, and post-course expectations Qualitative feedback indicated students were unclear about post-

course support, expected abilities post-/pre-course



Students: 125

Classes: 5

Locations: 4

Students: 1000

Classes: 47

Locations: 16

2017 2018


The Participant Learning Journey

Join Internal Program Hub

Skills Assessment

Registration Period Pre-Work Classroom

LearningProject

WorkPost-

Assessment

‣ Participants join Booz Allen’s internal messaging platform for access to all information and updates on the learner journey

‣Participant partakes in relevant skills assessment (DA1 and DS1) to determine current capability level and future learning path

‣Based on interest and location/ objectives/workstream, the participant enrolls in a program that matches their needs

‣Participant undertakes self-paced online modular content as pre-work in order to prepare for in-class training

‣Participant attends in-person classroom sessions. Selection of two formats (2 nights per week over 10 weeks or 8 weeks one full day per week)

‣Participants showcase their work in Capstone projects. Career managers and senior executives in attendance

‣Participant takes post assessment to quantify the knowledge uplift


Booz Allen’s Return on Investment

Sarah, Associate

● Demonstrated simplicity of performing

repetitive, predictable analysis in Python

for her client.

Brad, Lead Associate

● Implemented the usage of Python and

SQL at his client site and stood up a

data science practice

● Client excited by quicker turnaround

times and increased data processing

efficiency.

● Started the discussion about the type

of analysis that open-source software

can perform. The client moved forward

with standing up a robust architecture

to support predictive services.

Impact On Work:

Impact on Clients:

discover new talent pipelines by radically reskilling your ......booz allen’s data science journey...

Documents