data qc & archiving: overview and status seth mcginnis
DESCRIPTION
Data QC & Archiving: Overview and Status Seth McGinnis. 4 th NARCCAP Users Workshop NCAR Boulder, CO April 10-11, 2012. Outline. Current Archive Status QC Process Overview Derived Data Products Citations and Contributions. The Data Pipeline. Simulation Post-processing - PowerPoint PPT PresentationTRANSCRIPT
Data QC & Archiving:Overview and Status
Seth McGinnis
4th NARCCAP Users Workshop
NCAR Boulder, CO
April 10-11, 2012
Outline
• Current Archive Status
• QC Process Overview
• Derived Data Products
• Citations and Contributions
The Data Pipeline
• Simulation
• Post-processing
• Data transfer
• QC
• Publishing
Simulation Status
NCEP CCSM CGCM3 GFDL HadCM3
CRCM done done done
ECP2 done done setup
HRM3 done done done
MM5I done done running
RCM3 done done done
WRFG done done done
TMSL done done
Current Pipeline Status
• Ready to publish
CAM3 current 3D• QC in progress
HRM3-gfdl T5• Post-proc in progress
ECP2-ncep 2D• QC not started yet
MM5I-ccsm 2D fill-inMM5I-ccsm curr T5
• Needs post-process(in-house at NCAR)
ECP2 ncep, gfdl 3D ECP2 gfdl-future 2DCAM3 future timeslice
• Needs post-process(by modelers)MM5I ncep: misc. 2DMM5I ccsm-future: 3D
RCM3 all runs:
cli, clw, wa, misc. 2D
QC Overview
• Tally variables & files
• cf-checker
• Time coordinate continuity
• Temporal min/max/mean
• Time-space cross-sections
Example QC Plots
Fixing errors
• Metadata errors: fix
• Data gaps: fill with missing_value
• Bad values: replace with missing_value
• More serious errors: contact modelers
Derived Data Products
• Derived variables: tools on website
• Temporal averages & climatologies
• Regridded to 0.5-degree lat/lon grid
• Online GIS access via databasin.org
• New gridded daily obs: MICA
Bukovsky Regions
http://www.narccap.ucar.edu/contrib/bukovsky/index.html
• Descriptions
• Maps
• Mask Files
Citation & Contributions
• Data Citation: http://www.narccap.ucar.edu/about/citation.html
• Submit papers, presentations: http://www.narccap.ucar.edu/cgi/submit-paper.pl
• Sharing software: http://www.narccap.ucar.edu/contrib/tools/https://wiki.ucar.edu/display/narccaptopics/Tools
• DOI for the dataset: in progress
Questions?