batch process overview. batch processes 2 what do you mean by batch processing? allows for mass...

Batch Process Overview

Batch Processes2

What Do You Mean By Batch Processing?Allows for Mass Entry of DataAll Processing of the Data Happens Without Your

InteractionThe System Will Tell You When the Job Has

Completed and Provide You With Reports at the End of Each Process

Batch Processes3

What Can Be Input by Batch?

CAN Batch Load CANNOT Batch Load

Site information (all) Comments (free format text describing Special Information for Sites, Monitors, Audits, and Raw Data Values)

Monitor information (all) Technical systems audits

Raw data Exc. Event definitions

Precision & bias data EPA Concurrence (Regional user only)

Blanks data Primary Monitor Period designation for PM2.5

Batch Processes4

CDX

Production Tables – Production Status

Legend

AQS Batch Data Input Flowchart

Errors

Edit Error

Detail

Post

End

Correct

Raw

Edit/Load

Summary

Submit

Correct

Data

Load

File

Stats

CR

Post

Staging Tables

Production Tables - Preproduction Status

Y

N

Y

N

Stat

Eval

Scan

Report

Raw Data

Inventory

Batch

Report

Online

R = Relational Status

S = Stat Analysis

Maintain

Raw

Y

N

Batch Processes5

What’s the Big Picture?Collect SamplesFormat the DataSend the Data so AQS Can See it (CDX)Load the Data (Includes Error Detection)Correct Any Errors & ReprocessAnalyze and Approve Sample DataMake Available to Everyone

Happens Outside of AQS

Raw Data

Only

Batch Processes6

Where Do I Start?

Batch Processes7

Where To Go From Here

Files that You have Transferred to CDX

List of All Batch Processes run for Screening Group

Available ProcessesAvailable Reports

Batch Processes8

Batch Load: Use CDX to transfer files to AQS1. Start AQS, Login and select Screening Group.

2. Choose Batch Menu option

3. Use CDX button

37

Batch Processes9

Batch Load: Use CDX to transfer data from their PC to server (cont.)4. Enter user name, CDX password and Login

Batch Processes10

Batch Load: Use CDX to transfer data from their PC to server (cont.)5. Select AQS File Transfer

6. Click on Select and Specify files to collect from your computer

Batch Processes11

Batch Load: Use CDX to transfer data from their PC to server (cont.)

Batch Processes12


7. Click on Send to deliver all files to AQS. Close browser window.

Batch Processes13


8. After receiving email notice of delivery, click on Refresh CDX to see the newly loaded files.

Batch Processes14

File is on the server - We are finished with CDXNow, how to load

the file(s) to AQS?

CDX


Legend


Errors

Edit Error

Detail

Post

End

Correct

Raw

Edit/Load

Summary

Submit

Correct

Data

Load

File

Stats

CR

Post

Staging Tables


Y

N

Y

N

Stat

Eval

Scan

Report

Raw Data

Inventory

Batch

Report

Online


S = Stat Analysis

Maintain

Raw

Y

N

Batch Processes

15

CDX


Legend


Errors

Edit Error

Detail

Post

End

Correct

Raw

Edit/Load

Summary

Submit

Correct

Data

Load

File

Stats

CR

Post

Staging

Tables


Y

N

Y

N

Stat

Eval

Scan

Report

Raw Data

Inventory

Batch

Report

Online


S = Stat Analysis

Maintain

Raw

Y

N

Batch Load: Use Batch Load to Install Data

1. Select file (highlight) to load in CDX block2. Choose “Load File”3. Wait for email or Refresh in session block for

ERROR/COMPLETE status.

11

22

33

Batch Processes16

CDX


Legend


Errors

Edit Error

Detail

Post

End

Correct

Raw

Edit/Load

Summary

Submit

Correct

Data

Load

File

Stats

CR

Post

Staging Tables


Y

N

Y

N

Stat

Eval

Scan

Report

Raw Data

Inventory

Batch

Report

Online


S = Stat Analysis

Maintain

Raw

Y

N

Batch Load: Loading Raw DataPossible Results of Load

- COMPLETED load

- Load with ERROR

No data in Staging tables, Summary reports successes; next step Stat/CR

Data in Staging tables, Summary reports successes and failures; next step fix errors

Batch Processes

17

CDX


Legend

Errors

End

Raw

Edit/Load

Summary

Load

File

Staging Tables

N

Batch

Report

Online

N

Batch Load: Loading other than Raw Data

Possible Results of Load- COMPLETED load

- Load with ERROR

No data in Staging tables, Summary reports successes; End

Data in Staging tables, Summary reports successes and failures; next step fix errors

Batch Processes18

RecapSteps:

No Raw Data and No ErrorsCDX Load

No Raw Data and ErrorsCDX Load Correct Submit Correct Data

Raw Data and No ErrorsCDX Load Stat/CR Post

Raw Data and ErrorsCDX Load Correct Submit Correct Data

Stat/CR Post

Status = RStatus = S

Status = P

Batch Processes19

Caveats:A batch load job can end with a status of

“WARNING”The only way to see the warning message is to

look at the email job log. Example Warnings:

Urbanized Area Code replaced by Geospatial Lookup

Batch Processes20

Put it into Practice: Exercise 2.2

What is changing?Submit a file without logging into AQSNew CDX website (ENSC) & new CDX user IDFiles submitted to ENSC will be automatically

processed by AQSStat/CR is now part of LoadNew AQS batch formProcessing done by file, not by screening

groupReports are auto-generatedReports are consolidated

Batch Processes21

“Old” vs “New” process

AQS User

AQSCDX replacement

1st Q 2012AQS AutoloaderUser Decides Where to Stop

EN Node

CDX WebOld

pro

cess

New

pro

cess

Stage Load Post

UploadVia CDX Load StatCR Post

Batch Processes22

AQS Batch form (“new”)

Batch Processes23

ENSC websiteExchange Network Services Center (ENSC)The CDX replacementEN Username = your email address

Batch Processes24

ENSC -- file upload form

required

required

required

optional

optional

required for XML

Stage

Load

PostYes

No

File to Upload

Email address

AQS User ID

Screening Group

Stop on Error:

required

optional

Final Processing Step

XML Schema

File Type

XML

Flat

Card

One file only; Must be zipped

Default Final Processing Step is

Load.Default Stop on Error

is Yes.

Batch Processes25

batch process overview. batch processes 2 what do you mean by batch processing? allows for mass...

Documents