Quality Control

Data Processing OperationsScanning data capture and quality assurance

Quality in the Data Process

Geoffrey Greenwell, Data Processing AdvisorIPC

Quality in the Data PROCESS

P-R-O-C-E-S-S

Personal Process

Automated Process

P-R-O (TQM)

P ersonal Commitment to Excellence R eject Cynicism or Satisfaction O wn the process

Conceptual Overview

C-E-S-S (Six Sigma, Lean Manufacturing)

C areful Design E valuate Continually S hift Perspectives S helf Life

Conceptual OverviewForm Flow

InformationExformation

Data EntryManualScanning

EditsStructurePre-editConsistency

Tabulation controls

Data Archive

Form flow

•Flow Charting is a fundamental tool for careful design.•Flow charting is the mapping of the process.

•A flow chart is defined as a pictorial representation describing a process being studied or even used to plan stages of a project. Flow charts tend to provide people with a common language or reference point when dealing with a project or process. deming.eng.clemson.edu

•EXAMPLE

•Process Flow Chart- Finding the best way home•This is a simple case of processes and decisions in finding the best route home at the end of the working day.

Primary Symbols

Form FlowSoftware for flowcharting:

•ABC Flowcharter•Visio•Corel Flow•(Microsoft word has basic flow chart symbols)

Form Flow

External information flowExformationAll the processes involved in managing questionnaires andproducing progress reports.

Internal information flowInformationAll the processes involvedin managing the electronic processes and producing FEEDBACK.

Paper Flow Data flow

Ex-Form FlowLook at the “nodes”. In this case, the exchange of forms is a “pressure drop.”

1

1. Field office to data entry center.2. Center to Date entry group supervisor.3. Supervisor to data entry person4. Problems/trouble shooting.5. Storage6. Retrieval

2A

2B

2C

4

3

5

6

Ex-Form Flow

Every node is a potential loss of control in the process.

Ways to control:

•Operational control forms (electronic systems)•Progress reports•Assign responsible persons

Ex-Form Monitoring

Control Forms: Input and Output

Define a fundamental unit based on geographic criteria.

Census: Enumeration area=a box of forms=an electronic batchSurvey: A region=a time cycle=an electronic batch

• All forms are traced and verified to a master control form.• The flow of the form is followed through the phases of the data process.• The process has an audit trail all the way to the individual data entry station.

Ex-Form FlowA division of labor into logical, efficient and affordable processes designed to optimize the efficiency of the production line.

Labor Intensive Solutions

P=f(K,L) Classic production function.Capital and labor inputs

SpaceTrained personnel (division of labor)Low tech supplies: boxes, paper, labels

In-form flow

Network serversData entry stationsCablingPrinters

An electronic management system for:A. Processing the primary survey or census instrument.B. Provide a tool for managing the ex-formation.C. Provide feedback in the form of reports (end of information).

Capital Intensive Solutions

In-form flow                                                                                                                                                                                         

Form Flow (In or Ex?)•Transferring the box from the project vehicle to the forms depot.

•Designing a flow chart for a data entry program.

•Assigning a form to a data entry operator

•Scanning or keying in a form

•Two EA boxes are not closed well and the forms scatter during transport.•Backing up data on the central server.

•Supply clerk checking the inventory of paper; folders; labels and filling out a form to re-stock needed items.

Data Entry

Manual Entry vs. Scanning

The Great Debate…

$ pre unit of time to process

Manual Data Entry

• Cost Consideration

A. Training for all Clerks

B. Monitoring systems and supervisory time.

C. Verification of Clerk’s Work

D. Keying costs

Manual Data EntryDesign Considerations:

• Data File Structure Levels Multiple items vs. records

•Data Dictionary (Critical through all stages!)Good and consistent variable labelsLogical and efficient variable namesWell defined value sets (several value sets)

•Screen FormatsFont Size and typeSoft backgroundsField and text positioningField SizePhysical form emulation

• PathLogical PathItems vs. fieldsLevels

Manual Data Entry

• Skips To skip or not to skipCensus vs. Surveys

• On-line edits/Error Messages and Warnings

Census vs. Survey

CAPI (Computer Assisted Personal Interviewing)

Manual Data Entry

• Controls the process of carrying out the interview by the enumerator.• Removes the ex-form process by directly keying in responses into a portable computer.

CATI (Computer Assisted Telephone Interviewing)

• Controlling the data entry personnelEstablish objective measures (extract information from Log Files)

Manual Data Entry

Speed and accuracy (8000 keystrokes/hour)

• Heads up vs. Head down keying

• Census vs. Survey Constraints

Censuses: High Volumes (Time primary quality constraint)

Surveys: Low Volume (Place is the primary constraint)

Manual Data EntryVerification procedures:Verification is a duplication of the data entry process in order to compare two identical records for inconsistencies

Dependent vs. independent

Dependent verification duplicates the data entry process and compares the data files “on-line” and corrects the files when an error is encountered.

Independent verification is a complete re-entry of a form followed by a full comparison of the two data files.

Verification is dynamic and is adjusted to the learning curve. 100% at the outset and may drop to 2-3% at the end.

ScanningRefer to: A Comparison of Data Capture Methods by Sauer

Machine quality is very important.• Glass optics and color corrected instead of plastic optics

(light sensors and diffusion)• Resolution (DPI) 300• Bit depth: higher bit the better ability to interpret grayscales• Scanning speed• Optimal environment: 60-85 F and 40-60% relative hum.

Scanning

Rotary Scanners move the form. Best for censuses and surveys.Other Options to consider:•Automatic document feeder•Multi feed detector•Exit hoppers•Color bulbs•Image processing

ScanningCharacter reading: OCR, ICR and OMR

• OMR-Optical Mark Reading. Reads a mark from a questionnaire.

• OCR-Optical Character Recognition. Converts characters through photosensitive sensors and software enhancements.

• ICR-Intelligent Character Reading. ICR is pattern based character recognition and is also known as Hand-Print Recognition. (Software differences). Remembers patterns.

Note: OCR and ICR usually require “constrained handwriting” orBLOCK capital letters.

Scanning

Advantages and Disadvantages

Speed of process Loss of process controlTechnological Innovation Minimal technological transferMinimize human error Maximize machine error

Cost

World Bank CWIQ (Core Welfare Indicators Questionnaire)

ScanningQuality Issues

•Pencil type, paper jams, damaged forms•Accurate character recognition dependent on form quality and image•Field level accuracy vs. character level accuracy•Machine Maintenance•Software deficiencies (Voting)•KFI (Key from image) for character correction•KFP (Key from paper) for form and field correction•Confidence level reports (recognition rates)•See page 18, Sauer for Quality Control issues

Coding

Coding of open ended questions like: Occupation and activityrequire coding.

In Scanning this can be done from the image. In Manual entry it is usually the first step.

It requires its own process and supervision.In Scanning it can be seen as a parallel operation.In Manual Entry it is linear.

Now that you have: A well designed dictionary with the simplest and most efficient structure and well defined variable labels with easy to use variable names and well defined value sets clearly designed, simple and user friendly screens emulating the forms and flowing logically with programmed skips and have clear interactive messages should you use them and have defined productivity targets for your census or survey and a system to objectively measure productivity and reward accordingly and verified the work and finally rigorously subjected it against the PROCESS rule of quality…

you still need to check for errors.

Edit FlowI. Verified

Files

II. Structure

Consolidate

II. Pre-Consolidate

III. Consistency

IV. Pre-Tabulation

Data Processor Statistician Data Processor Analyst

Structure Edit Controls:• Performed after verification

• Control totals used to check completeness

– File totals compared with manual counts

• Corrections done with questionnaires

• Limit checks to rendering questionnaires clear enough

for computer processing (processability) only.

Pre-edit Consolidation Controls:

• Follow pre-established geographical priorities

• Check control totals

• Use operational control data base

• Avoid geographic coding (geocode) conflicts in

joining files

Consistency Edit Controls:• Develop consistency specifications

• Prioritize variables

• Monitor corrections

• Use control tabulations

• Re-run output file

Pre-tabulation Consolidation Controls:

• Consolidate to facilitate tabulation

• Check control totals for each record type

• Use standardized forms for operational control

• Avoid geocode conflicts in joining files

Tabulation Controls:• Computer Program Specifications

• User Approval of Tables

• Tables Grouped by Characteristics

• Tables Checked Against Control Totals

• Control Tables Show Weighted and Un-weighted Numbers

• Geographic Subtotals Match

• Standardized Control Forms for Production Control

• Final Table Review

• Data Dissemination to User Community

Data ArchivingData and Metadata

metadata are often called "codebooks“Metadata is the data which defines the data

http://www.icpsr.umich.edu/DDI/ORG/index.html

DDI-Data Documentation Initiative

The definition of an international standard to defineMetadata in the social sciences.

Data Archiving

The DDI defines a hierarchy of information related to a Census or survey.

The DDI defines these by using XML (Extensible Mark up language)

XML defines document tags much the same way as HTML.

Example of Codebook Structure:

http://www.icpsr.umich.edu/DDI/CODEBOOK/codedtd.html

Data Archiving

NESSTAR is an example of a web based distributed DDI compliant system.

Data Archiving

All processes need to be documented. This includes:CodebooksData entry manualsProgram documentationEdit programsImputation rulesTracking/process manuals

Final ConceptsLean Manufacturing: A quality control system for monitoringprocess flow.

Six Sigma: A statistical system developed by Motorola to establish process problems and error tolerances and methods to correct.

Final ConceptsLean Manufacturing:

•Workplace organization•Standardizing work/work stations.•Division of labor to increase process flow.•JIT-Just-in-Time delivery•Pull systems

Six Sigma:•Means six deviation tolerance for error: 3.4 error events out of one-million•Measurement (quantifiable) system for process improvement.

Final Concepts

P-R-O (TQM)

Personal Commitment to Excellence Reject Cynicism or Satisfaction Own the process

Conceptual Summary

C-E-S-S (Six Sigma, Lean Manufacturing)

Careful Design (Flow Charts) Evaluate Continually (Six Sigma) Shift Perspectives (Lean Manufacturing)Shelf Life (Data Archive)

From John Henry

The man that invented the steam drillThought he was mighty fineBut John Henry made fifteen feetThe steam drill only nine, Lord, LordThe steam drill made only nine.

John Henry hammered in the mountainHis hammer was striking fireBut he worked so hard, he broke his poor heartHe laid down his hammer and he died, Lord, Lord.He laid down his hammer and he died