interfaces eo data with atmospheric and land surface model: progress report 2
DESCRIPTION
Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2. Liang Feng , Paul Palmer . Project Tasks. Reference OSSE system High-resolution model CO:CO2 ratios. Observing System Simulation Experiment Tool. I. Progress overview. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/1.jpg)
Interfaces EO data with Atmospheric and Land Surface Model: Progress
report 2
Liang Feng, Paul Palmer
![Page 2: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/2.jpg)
A. Reference OSSE systemB. High-resolution modelC. CO:CO2 ratios
Project Tasks
![Page 3: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/3.jpg)
I. Progress overviewOverall Aim: Developing a reference OSSE simulation for community researchers to evaluate impacts of space-borne atmospheric composition measurements on surface flux estimates. Outputs:
1). OSSE framework with flexiable module and libraries. Status: Done.
2). One complete example OSSE system for 1) simulating OCO-like XCO2 observations; and 2) estimating regional CO2 fluxes by assimilating XCO2 observations using an Ensemble Kalman Filter (EnKF).
Status: Done . 3. Detailed documents. Done. 4. EOF and Visualization Tools. Done.
Observing System Simulation Experiment Tool
![Page 4: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/4.jpg)
II. Download and Installation Codes and documents:
1) Packages for PyOSSE and EOF visualizations can be downloaded from http://xweb.geos.ed.ac.uk/~lfeng 2) Installation guides has also been provided, covering how to:
• unzip archive; • set python search path;• build shared libraries;• test codes.
3) documents and references can be found on-line or from local directory.
Data• Being required mostly by the example OSSE system.• Covering climatology for cloud, aerosol PDF , satellite orbit,
instrument averaging kernel, and GEOS-Chem model outputs. • Being stored as ASCII text files, netCDF files and BPCH files in
www.esa-da.org/data/otool_data.tar.gz.
![Page 5: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/5.jpg)
III. PyOSSE framework
Data flow and directory structureClasses IO ModulesConfiguration files
![Page 6: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/6.jpg)
Obs+random errors
Flux Forecasts
Obs operator
Forecast(3-D concentrations)
CTM
Prio
r + e
rror Posteriori +
error
Observation Simulation EnKF
assimilation
ETKF Model ObsEnsemble
Ensemble forecasts(3-D fields)
Surface flux Ensemble CTM
Obs operator
Data flow and directory structure
![Page 7: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/7.jpg)
observation (sample/convolve)
instrument (ORB, AVK. CLD …)
example (OSSE for OCO)
surface (flux ensemble)
atmosphere (CTM outputs)
util (lib)
enkf (stv and etkf solvers)
PyOSSE package has over 90 python/fortran modules, stored in 8 subdirectories:
![Page 8: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/8.jpg)
Classes:
We have defined about 40 python classes as containers for data and functions of model objects
Measure
Transport
Top-down estimate
Atmosphere
Observation
Surface flux
Flux modelinventory
Met files
![Page 9: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/9.jpg)
class ens_stat_cl: Class for state vector used to assimilate observations Members: (total of 44) ------------------------------------------------------------- 1. attr_dict:<dict>: attributes 2. step_tag_lst:<list>: ID for run step # variables for current assimilation window 3. wnd_mean_x0:<array, (wnd_nx)>: prior coefficient values for lag window 4. wnd_mean_x:<array, (wnd_nx)>: posterior coefficient values for lag window 5. wnd_dx:<array, (wnd_nx, wnd_ne)>: perturbations ensemble for coefficients 6. wnd_inc_m:<array, (wnd_ne>: increment matrix for ETKF data assimilation 7. wnd_xtm: <array, (wnd_nx, wnd_ne)>: transform matrix for ETKF data assimilation 8. wnd_xinc: <array, (wnd_nx)>: increment for ETKF data assimilation # current analysis increments 9. inc_m:<array, (wnd_nx)>: increment matrix for current step 10. xtm: <array, (wnd_nx, wnd_ne)>: transform matrix for current step 11. xinc: <array, (wnd_nx)>: increment for ETKF data assimilation
Functions (total of 14) ========================================================== 1. __init__: initialization 6. construct_tcor_matrix: Create temporal correlation matrix 7. add_new_x_to_window: Add new apriori to assimilation window 10. do_assim: assimilate observations and update state
Example: class for state vector
![Page 10: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/10.jpg)
Configurable IO modules• PyOSSE relays on many pre-defined data sets such as satellite orbit,
instrument sensitivity, and cloud PDF etc. • PyOSSE needs mechanism to exchange data with CTM. • Data is usually given in different formats, and often needs pre-
processing. • Configurable IO modules are designed as bridges between host
classes and data files.• These modules are easy to be re-configured or even be replaced.
Var-listVar-dictfdescfopenfclosefwritefread
Def-Var-ListDef-Var-dict Create-fdesc
OpenfileReadfilewritefileClosefile
.
data
Host class IO module Disk files
![Page 11: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/11.jpg)
Configuration filesWe have developed a new type of text-based menu files to:
1. Numeric or string parameters as function inputs (traditional).2. Object as parameters such as list, functions (new) 3. Object creater (new)
#MENU (cloud)#================================================name|cloud path|$DATAPATH$/clim_dat/flnm|meancloud_frac_XMONTHX_landsea.datdict|__load:$MDPATH$.cloud_file_m:cld_varname_dictfopen|__load:$MDPATH$.cloud_file_m:open_cloud_filefread|__load:$MDPATH$.cloud_file_m:read_cloud_filefclose|__load:$MDPATH$.cloud_file_m:close_cloud_filefget|__load:$MDPATH$.cloud_file_m:get_cloud_datakeywords|__dict:Nilfclass|__load:$MDPATH$.cloud_m:cloud_cl#MEND
Example: vob_def.cfg for observation simulations allows users to choose classes for satellite orbit, averaging kernel, cloud/AOD PDF , and sampling etc
![Page 12: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/12.jpg)
#MENU (data random sampling)#=======================================================name|sample# cloud sampling fcld_sample|__load:$MDPATH$.cloud_m:cloud_samplefcld_penalty|Nonefcld_keywords|__dict:Nil
# AODfaod_sample|__load:$MDPATH$.aod_m:aod_samplefaod_uplimit|0.3faod_keywords|__dict:Nil
# landcoverflc_sample|__load:$MDPATH$.landcover_m:sample_lcflc_stype_lst|__load:$MDPATH$.ak_file_m:ak_lc_stype_lstflc_keywords|__dict:Nil
#MEND
Menu term (sample) in vob_def.cfg
![Page 13: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/13.jpg)
Menu term (averaging kernel ) in vob_def.cfg
#MENU (averaging kernel)#===================================================== name|ak path|$DATAPATH$/oco/oco_ak/flnm|aknorm_XSURFACEX.XEXTXviewmode_dict|__load:$MDPATH$.ak_file_m:def_ak_viewmode_dictstype_dict|__load:$MDPATH$.ak_file_m:def_ak_surf_type_dictfopen|__load:$MDPATH$.ak_file_m:open_ak_filefread|__load:$MDPATH$.ak_file_m:read_ak_filefclose|__load:$MDPATH$.ak_file_m:close_ak_filefget|__load:$MDPATH$.ak_file_m:get_ak_datakeywords|__dict:Nilfclass|__load:$MDPATH$.ak_m:ak_cl#MEND……
![Page 14: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/14.jpg)
IV. OSSE Example system
Observation simulation Inversion
![Page 15: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/15.jpg)
Observation simulation (example/obs_simulation):
1. Generate dummy observationby using gen_dummy_obs.py (and vod_def.cfg)
2. Generate model observationby sampling actual model outputs (gen_sat_obs.py and sob_def.cfg)
![Page 16: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/16.jpg)
#MENU (class for model profile)#================================================name|fc_prof# model output path|$DATAPATH$/gc_std/# file name (empty to use defaults in )flnm|'ts_satelliteXEXTX.XYYYYXXMMXXDDX.bpch'fdiaginfo|'diaginfoXEXTX.dat' ftracerinfo|'tracerinfoXEXTX.dat' vname|single_profileext|'.ST001.EN0001-EN0002'# output access fopen|__load:ESA.util.gc_ts_file_m:open_ts_filefread|__load:ESA.util.gc_ts_file_m:setup_daily_profilefget|__load:ESA.util.gc_ts_file_m:get_mod_gpfpres|__load:ESA.util.gc_ts_file_m:get_mod_presfclass|__load:ESA.atmosphere.ts_slice_m:gc_slice_cl
#MEND
GEOS-Chem outputs can easily be replaced by user’s CTM simulations by changing sob_def.cfg
![Page 17: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/17.jpg)
Inversion (example/obs_simulation): 1. Settings: default settings are defined in osse_def.cfg, configuring classes for state vector, flux, perturbation ensemble, prior flux, observations, model forward simulation, and model outputs sampling etc.
#MENU (CTM for single tracer run)name|ctmrunpath|$FCRUN_ROOT$datapath|$FCRUN_DATAPATH$tra_st|1tra_end|2finput_gen|__load:ESA.example.enkf_oco.input_geos_gen:create_new_input_fileext|ST001.EN0001-EN0002runtype|3runscript|rungeos.shfclass|__load:ESA.example.enkf_oco.run_geos_chem:geos_chem_cl#MEND
Example: CTM defined in osse_def.cfg
![Page 18: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/18.jpg)
2) Results:
We have a rich set of outputs for inversion results as well as diagnostic data
For example: Posterior fluxes
![Page 19: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/19.jpg)
For example: Posterior model XCO2
![Page 20: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/20.jpg)
V. EOF and visualization
The pyeof module
Visualization package
![Page 21: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/21.jpg)
pyeof• Pyeof defines a class eof_cl to find EOFs for temporal and spatial
data sets. • It uses SVD technique to find eigenvalues and eigenstate for
covariance matrix. • It is easy to use, and also provides functions for visualization of
EOFs and PCs
EOF1
![Page 22: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/22.jpg)
Visualization
• We have developed a wxpython-based GUI application for quick visualizations including 2D plots and animations.
• It also has a python shell to use a rich set of existing visualization tools.
• We have also provided functions for common data processing.
![Page 23: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/23.jpg)
![Page 24: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/24.jpg)
![Page 25: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/25.jpg)
VI Conclusion OSSE tool and visualization package are ready for public
release.
This framework is simple and flexible, due to the adopted algorithm (EnKF), and the way to implement it (classes, configurable IO modules, and enhanced menu functions …).
It has been successfully used to digest GOSAT observations.
We are making further developments.
![Page 26: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/26.jpg)
B. Hi-resolution modeling
![Page 27: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/27.jpg)
I. Motivation Interpret and digest future satellite (like OCO-2) observation of
atmospheric CO2 concentrations: Model errors (transport and representation errors) have significant
impacts on top-down estimates of surface fluxes. Higher model resolution usually means better transport modelling, and
smaller representation errors.
Provide benchmarks for coarser simulations or nested model simulations.
Itself is one by-product of our efforts to develop EnKF approach based on nested GEOS-Chem transport model.
In-cooperate other process-based models (such as plume model) into GEOS-Chem (Gonzi et al, University of Edinburgh).
Due to the finer temporal and spatial resolutions, It is more sensitive to
the details of ‘small scale’ processes.
![Page 28: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/28.jpg)
II. Challenges:
Availability of High-resolution emission inventories
• 1x1 monthly fossil fuel emissions (Oda et al).
• 1x1.25 3-hourly CASA biospheric CO2 exchanges (R. Kawa , J. Collatz, and D. Liu)
• 1x1.25 daily biomass burning (R. Kawa , J. Collatz, and D. Liu)
• 4x5 monthly oceanic surface CO2 flux (Takahashi et al)
![Page 29: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/29.jpg)
Programming issues
1. Memory:
In particular, stack size is limited by most of fortran compiler. Modifications have been made over GEOS-Chem v8.02 to limit the use of common block and avoid static arrays in modules. 2. Parallelisation:
Mainly we have to reschedule parallelisation in tpcore modules, and change functions in dao_mod.f .
3. Minor issues: Mainly they are associated with diag outputs.
![Page 30: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/30.jpg)
Results
Comparison with standard GEOS-Chem 4x5 simulations
CO2 /ppm (ML1-8) CO2/ppm (ML1-8)
Range: 374 – 415 ppm Range: 377 ppm – 403 ppm
![Page 31: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/31.jpg)
Comparison with nested simulation
Global 0.5x0.666 Nested 0.5x0.666
Nested model being used in UK GAUGE project
CO2 /ppm (ML1-4) CO2 /ppm (ML1-4)
![Page 32: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/32.jpg)
Comparison with GEOS-Chem drived by ECMWF winds
GEOS-5 winds ECMWF winds
ECMWF Met fields may have stronger vertical transport across boundary layer.
CO2 /ppm (ML1-8) CO2 /ppm (ML1-8)
![Page 33: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/33.jpg)
Comparison to averaged aircraft profiles
TABATINGO SANTAREM
RIO BRANCO ALTA FLORESTA
N = 11 N = 7
N = 11 N = 11
--- GEOS-5 PBL
![Page 34: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/34.jpg)
Summary Native resolution simulation provides:
a) More realistic descriptions of observed variations
b) Benchmark for lower-resolution model
c) Framework for including detailed process models
Our comparisons also reveal some possible model errors.
We are now working on flux inversions for selected regions • We have done some OSSEs for Astrium Project • We are working on digesting ICOS data.
![Page 35: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/35.jpg)
Spare slides
![Page 36: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/36.jpg)
Top-down Optimal surface flux estimation
x: regional surface fluxesyobs: measurements of atmospheric concentrations. H: Jacobian (CTM model).
)( fobs
fa HxyKxx
K=PfHT(HPfHT+R)-1 – Kalman gain matrix.Pf: a priori uncertainty matrix R: observation error matrix
Posteriori Priori gain observation model
Data Assimilation Module (Ensemble Kalman Filter)
![Page 37: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/37.jpg)
Ensemble Approach
Key features: (Feng et al., 2009 ; 2011)• Represent a-priori uncertainties by an ensemble of flux
perturbations. )T
• Use a lag windows to limit computational costs. Any emission will only be constrained by observations within a following limited time period. After that period, it is considered to be well-known.
• Project the ensemble of perturbations together with prior estimates into the observation space using CTM.
• Use Ensemble Transform Kalman Filter (ETKF) to determine posterior fluxes, and the associated uncertainties from digesting observations.
![Page 38: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/38.jpg)
A. State vector and ensemble representation of its uncertainties
),,(),,(),,( 0 tyxBFctyxFtyxF l
ll
Surface fluxes Prior estimates of surface fluxesBasis functions for pulse-like flux perturbations,
:coefficients to be estimated: State vector: , , , …, ]; Perturbations: , , , …, ]Error covariance: Full or partial representation: , , , …, ],
![Page 39: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/39.jpg)
We use region_map_c.py to defined as 8-day surface CO2 flux perturbations over 144 global regions.
We use gen_region_pb_flux_m.py to generate BFs from biospheric GPP map and multi-layer maps for 144 regions.
![Page 40: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/40.jpg)
B. Ensemble forecasts
• We change IO function in pb_flux_c.py to generate the ensemble of flux perturbations according to the defined basis functions and pre-defined uncertainties.
• Functions in ctm_config_c.py and ctm_restart_c.py are override to support GEOS-Chem tagged runs to project flux perturbation ensemble to the ensemble of atmospheric tracer concentrations.
C. Projection of ensemble forecasts to the observation space (ctm_world_c.py)
![Page 41: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/41.jpg)
D. Inversion algorithm:Ensemble Transform Kalman Filter (ETKF)
• A posteriori and the associated uncertainties are simultaneously calculated using SVD (or sparse-matrix LU) technique.
+R]-1
TTTT=[1+ -1
• Corresponding updates are made to the ensemble of model 3D concentrations to retain the contributions of fluxes outside the assimilation (time) windows (Feng et al, 2009)
![Page 42: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/42.jpg)
Algorithm testing: Comparison with the LSCE 4d-var system
In the comparison experiments, we have assimilated the same ACOS-GOSAT B210 XCO2 retrievals over 2009 and 2010, but using different approaches: EnKF (UoE) and 4d-var (LSCE)
![Page 43: Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2](https://reader035.vdocuments.site/reader035/viewer/2022062323/568160dc550346895dd00b6d/html5/thumbnails/43.jpg)