data collection through analysis in the u. s. army’s enlisted common soldier task (ecst) project...

Data Collection Through Analysis in the U. S. Army’s Enlisted

Common Soldier Task (ECST) Project

Johnny J. Weissmuller Metrica

Bob GoscInstitute for Job & Occupational Analysis

IMTA - 11 Nov 1999

Data Collection Though Analysis 2

Background The Institute for Job and Occupational

Analysis (IJOA) was incorporated in 1993 as a not-for-profit agency pursuing scientific and educational goals in OA .

IMTA - 11 Nov 1999


Background Although IJOA undertakes operational

endeavors in the OA community, the goal is to develop better methods and tools -- and spread these (not project data) to academic, government, and private sector users around the world.

IMTA - 11 Nov 1999


U.S. Army Enlisted Common Soldier Task (ECST) Project

Goal: Survey all Enlisted Grades, across diverse Military Occupational Specialties (MOSs), to identify COMMON training needs

Output: to support the Army-wide Critical Task Selection Panel in revising the U.S. Army’s Soldier’s Handbook

IMTA - 11 Nov 1999


ECST Technological Challenges 847 Items, 47 Duties,

20,000 cases worldwide Army’s need for new

survey program features:– Sort by Duty Importance

– Screen by Skill Level

– Modify by Position

Using IJOA’s “Author” system for generating this AND 8 clones for TE surveys

IMTA - 11 Nov 1999


Stages in the Occupational Analysis (OA) Process

Inventory Development Survey Preparation Field Administration Data Aggregation Data Processing Analysis, Report

Writing, and End-User Assistance

IMTA - 11 Nov 1999


Inventory Development

Word 97 used to integrate task lists from diverse sources

Word 97/email used to coordinate around the world (track revisions)

Word 97 used to note target skill levels, source citations, etc.

IMTA - 11 Nov 1999


Word 97 to HTML Format

Administration NC Complete essential administrative support requirements SL5NC Develop unit inspection checklist 1SGNC Update unit special activities calendar 1SGNC Verify transmittal of documents to finance (leave/allotment/pay

elections)1SG

Awards and Decorations NC Maintain order of merit list for schools 1SGNC Manage awards and decorations program SL5NC Write an award recommendation SL4 Casualty Process C Conduct by-name casualty reporting SL3,4C Conduct mortuary affairs operations SL1NC Monitor organizational casualty reporting system SL5 Communicate C Analyze contents of communications

(correspondence/messages/directives) for actionSL2

C Apply an automated Signal Operation Instruction (SOI) SL1C Communicate via a tactical radio SL1

IMTA - 11 Nov 1999


Survey Instrument Preparation Word 97 documents saved in

Hyper Text Markup Language (HTML) format

Converted Task & Duty Statements using HTM2Flat

Converted Collateral data using HTM2Data

OASurv Electronic Survey Administration disks ready 15 minutes after receiving “GO” email.

IMTA - 11 Nov 1999


Word 97 to HTML Format<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=windows-1252"><META NAME="Generator" CONTENT="Microsoft Word 97"><TITLE> </TITLE></HEAD><BODY>

 <TABLE BORDER CELLSPACING=1 BORDERCOLOR="#000000"CELLPADDING=7 WIDTH=510><TR><TD WIDTH="7%" VALIGN="TOP"> </TD><TD WIDTH="80%" VALIGN="TOP">Administration</TD><TD WIDTH="13%" VALIGN="TOP"> </TD></TR><TR><TD WIDTH="7%" VALIGN="TOP">NC</TD><TD WIDTH="80%" VALIGN="TOP">Complete essential administrativesupport requirements</TD><TD WIDTH="13%" VALIGN="TOP">SL5</TD></TR>

IMTA - 11 Nov 1999


HTML Format to atCODAP

IMTA - 11 Nov 1999


HTML Format to OASurv/CODAP/GenSurv

1 A0001 Complete essential administrative support requirements 1 A0002 Develop unit inspection checklist 1 A0003 Direct preparation of "daily" staff journal for R&D 1 A0004 Maintain <unit reading> files 1 A0005 Make recommendations regarding report of survey 1 A0026 Verify transmittal of documents to finance 2 A0026 (leave/allotment/pay elections) 1 B0027 Analyze contents of communications 2 B0027 (correspondence/messages/directives) for action 1 B0028 Apply an automated Signal Operation Instruction (SOI) 1 B0029 Communicate via a tactical radio 1 B0067 Track administrative/Standard Installation Division 2 B0067 Personnel System (SIDPERS) actions for medical 3 B0067 evacuations of subordinate soldiers 1 B0068 Write a Standing Operating Procedure (SOP) 1 C0069 Assist commander to develop training schedules using the 2 C0069 standard Army Training System (SATS) 1 A ADMINISTRATION 1 B COMMUNICATE 1 C COMPUTER OPERATIONS

IMTA - 11 Nov 1999


Survey Instrument Preparation The changing abilities of

the OASurv Program Allowing many users on

a single diskette requires effective virus checking in the field

Capturing a “computer inventory” to determine what computer platforms in what quantities are in the field

IMTA - 11 Nov 1999


OASurv - The Survey Engine Upgrades & Support Programs

Extract “Target Skill Level” for OASurv & reports (HTM2Data)

Format “Target Skill Level” information for new OASurv Logic (PlusOne)

Rate Duty Importance, sequence & screen tasks based on those ratings

IMTA - 11 Nov 1999


HTML Format to OASurv/atCODAP Data List

IMTA - 11 Nov 1999


Field Administration Master Disk sent to a

commercial duplicator who copied, labeled, and returned 20,000 disks within 72 hours.

Disks shipped to 60 locations worldwide.

APOs Express Mail where available and Priority Mail for others.

FedEx for normal packages

IMTA - 11 Nov 1999


Field Administration Returns - Job Incumbent Surveys

19,600 surveys distributed across all components

7,689 disks returned

6,682 usable cases returned

Percent of distributed surveys returned

• Active 67.27%• Guard 23.71%• Reserves 26.76%

IMTA - 11 Nov 1999


Field Administration Training Emphasis (TE) Surveys

IMTA - 11 Nov 1999


Data Aggregation

Upload Spread across many PCsCombined Respondent Data, Logs, & Computer InventorySeparated & Linked Write-In-Text (*.WIT)Later employed an automated reader

IMTA - 11 Nov 1999


Data ProcessingData Processing was accomplished by atCODAP, a commercially available PC version of the CODAP system – i.e., the Comprehensive Occupational Data Analysis Programs.

Microsoft Access data bases and reports, along with other products were prototyped using partial data as “status reports” four times prior to the final cutoff of survey returns.

All final atCODAP products, including Microsoft Office export products from Access, were generated within 96 hours after data cutoff. This included subsampling for 72 Training Emphasis Inter-rater reliability analyses and job descriptions for 72 groups reporting annual frequency and percent members performing.

IMTA - 11 Nov 1999


Analysis, Interpretation, Report Writing & Aiding End-Users

•atCODAP Programs and reports were used as “Data Reduction Tools” and “Audit Tools”

•Summary Data Vectors were exported to Access using the OADecks program

•Tasks & Duty Affiliations were migrated to Access

•Access was used to produce “Reference Reports”

•Access was used to produce “Focus Reports”

•Access Forms were used to interactively capture decisions of Army Researchers and finally, of top Army policy makers.

IMTA - 11 Nov 1999


IMTA - 11 Nov 1999


Conclusions Required Technology

is “Commercial Off-The-Shelf” (COTS) or Government-owned software

Logistics have been optimized for rapid, large-scale usage

Psychometric & Human Factors Issues have been addressed

IMTA - 11 Nov 1999


Any Questions?

data collection through analysis in the u. s. army’s enlisted common soldier task (ecst) project...

Documents