use of scanning technology for data capture

1

Use of scanning technologyfor data capture

ICR System(Intelligent Character Recognition)

Information and Communication Technology Center

National Statistical Office (NSO)

Thailand

2

ICR SYSTEM

It is an automatically Data Entry by Scanning

questionnaires which can be evaluated

handwritten numeric / alphanumeric characters

(ICR) or handwritten optic marks (OMR) and

conduce to Digital Data before processing on

others Computer System.

3

ICR System

ICR System in NSO (Thailand) can be

divided into 2 parts :

TELEform Software System.

ABBYY Software System.

4

NSO provided ICR System to process

the first Project of Population Census in 2000.

By scanning the 16 million households

(16 million Forms) which spent only 8 months

to process the raw data instead of more than

1 year by using Key in Data System.

ICR for The Population Census 2000

5

ICR for The Agricultural Census 2003

NSO hired ABBYY Software to process about 25% of The Agricultural Census 2003 that has totally 5.7 million

households (24 million forms).

6

After processing The PopulationCensus 2000,some survey projects have

been used ICR System such as TheLabour Force Survey (monthly report),

The Household Manufacturing Survey etc.

Other Survey Projects

7

TELEform Hardware System

• NetServer for TELEform Server 1 Unit

• NetServer for Database Server 1 Unit

• Reader Modules Workstations 21 Unit

• Verifier Modules Workstations 30 Unit

• Scanner Control Workstations 6 Unit

• Scanner Fujitsu M4099D 6 Unit

8

TELEform Software System

TELEform Software is selected for ICR System which is used for Forms and Documents Processing. It is the product of TELEform Cardiff

Software , Inc. USA.

TELEform 6.2 Elite Enterprise Edition

Components: TELEform Designer.

TELEform Reader.

TELEform Verifier.

9

ABBYY Hardware System

• IBM Server X Series 225 1 Unit

• Correction Station 1 Unit

• Verifier Modules Workstations 25 Unit

• Scanner Control Workstations 4 Unit

• Scanner Fujitsu M4099D 4 Unit

• Storageflex LT707 1 Unit

10

ABBYY Software System NSO provided ABBYY Software which is the product of Russia.

ABBYY FormReader 6.0 Enterprise Edition

Components: ❑ Form Design

❑ Administration Station

❑ Recognition Station

❑ Correction Station

11

TELEform / ABBYY Designer Function

• To create template form.

• Fix value of data type properties in each field of template form on questionnaire.

12

TemplateQuestionnaire

13

TELEform Reader / ABBYY Administration Function

• To evaluate the questionnaires .

• Export the corrected questionnaires to a data file.

• Send the unclear questionnaires to TELEform / ABBYY Verifier Function for correcting and transferring the corrected questionnaires to a data file.

• Store scanned images.

14

TELEform / ABBYY Verifier Function

• To correct questionnaires that contain mismarked or illegible fields.

• The corrected questionnaires are automatically exported to a data file.

15

BasicScript Programming Language

Script Language use to add extensive features similar to Visual Basic.

• To develop Form Script during Evaluation and Correction.

• To develop Export Script during Data Export.

16

Step of ICR System in NSO Scan and Forms Distribution : The questionnaires are scanned in each Block / Village and created Multi Page Image Files in local hard disk.

Forms Evaluation : The questionnaires images are evaluated. The corrected questionnaires which skipped Verifier Workstation and directly exported to Database server.

17

The questionnaires are scanned.

18

Scan and Forms Distribution

Questionnaires Scanned Image files

19

Step of ICR System in NSO (cont.)

Forms Verification : The unclear questionnaires are needed review and corrected it in Verifier Workstations before transferring to Database server.

Data Export : - Link a data file from Database server to IBM Mainframe System. - Store Scanned image files to CD.

20

Unrecognized fields must be corrected at Verifier Workstations.

21

Forms Verification and Data Export

Storage (Images files)

Database Server

CD

Export data

for processing

Verify

22

Questionnaires

CD

Storage(HD 880

GB)

Mainframe

HP Server

COMPAQ Server

Processing(Editing & Reporting)

Scanners 4

unit

IBM Server

controllerPC 6 unit

30 unit

PC 4 unit

controller

21 unit

Verifications

Readers

Correction 1 unit

station

25 unit Verifications

-Administration-Export-Recognition

ABBYY Software

TELEform Software

ICR Link System

CD

Questionnaires

Transfers

- Backup Data

- Software - Database

S Scanners6 unit

24

Specific Questionnaire for ICR System

The questionnaires must be professionally printed in a specific colour field boxes

(blue,green,red).The scanners are recognized the questionnaires characters / numerics and drop out field boxes

colour.Test questionnaires to make sure that can process it with speed and accuracy.

25

ICR Benefits

Reduce Cost.

Reduce Time. Efficient Data Capture.Increase Data Accuracy.