concept of operations template - gri.msstate.edu · web viewthe format of this document follows...

Capabilities Description for RPC

CAPABILITIES DESCRIPTION

FOR

RAPID PROTOTYPING CAPABILITYFOR EARTH-SUN SYSTEM SCIENCES

Version 1.0

By RPC Project TeamMississippi State University

Robert Moorhead, P.I.David Shaw, co-P.I.

May 3, 2006

of 24


TABLE OF CONTENTS

1. Scope....................................................................................................................31.1. Document security...........................................................................................31.2. Identification....................................................................................................31.3. Document overview.........................................................................................31.4. System overview..............................................................................................32. References............................................................................................................62.1. Referenced documents.....................................................................................62.2. Acronyms and Definitions...............................................................................63. Current situation..................................................................................................73.1. Background, objectives, and scope..................................................................73.2. Operational policies and constraints................................................................83.3. Description of current situation.......................................................................83.4. Modes of operation for the current situation...................................................93.5. Users and other involved personnel.................................................................93.6. Support environment.......................................................................................94. Justification for and nature of changes................................................................94.1. Justification of changes....................................................................................94.2. Description of desired changes......................................................................104.3. Priorities among changes...............................................................................114.4. Changes considered but not included............................................................115. Concepts for the proposed system.....................................................................115.1. Background, objectives and scope.................................................................115.2. Operational policies and constraints..............................................................135.3. Description of the proposed system...............................................................135.4. Modes of operation........................................................................................195.5. User classes and other involved personnel....................................................195.6. Support environment.....................................................................................196. Operational scenarios.........................................................................................196.1. Scenario for idealized experiment evaluating data sources...........................206.2. Scenario for idealized experiment evaluating models...................................217. Summary of impacts..........................................................................................227.1. Operational impacts.......................................................................................227.2. Organizational impacts..................................................................................227.3. Impacts during development..........................................................................238. Analysis of the proposed system.......................................................................238.1. Summary of improvements............................................................................238.2. Disadvantages and limitations.......................................................................24

of 24


1. Scope

The Rapid Prototyping Capability (RPC) is being developed by the Mississippi Research Consortium (MRC) and its partners under contract to NASA Stennis Space Center.

Consider the following definitions in this document.

“Current” means as of Fall 2005.

The “proposed system” will be delivered by December 2007, and thus, will be operational by January 2008.

The development period (20062007) will include infrastructure development and performing of example experiments that will validate the RPC requirements and verify the infrastructure implementation.

1.1. Document security

This document is not restricted.

1.2. Identification

This document is identified by the title and effective date.

1.3. Document overview

This document describes the capabilities of the Rapid Prototyping Capability. The format of this document follows the outline of a Concept of Operations document, per IEEE Standard 1362-1998. Major sections are (1) a description of the situation as of Fall 2005, (2) justification for change, (3) operational concepts for a proposed Rapid Prototyping Capability, planned for January 2008, that includes both the activities of people and supporting technology, (4) idealized operational scenarios for classes of rapid prototyping experiments, (5) a summary of the impacts of the proposed system, and (6) an analysis of the improvements and limitations of the proposed system.

1.4. System overview

Within the systems engineering approach to NASA’s Earth Science Application Plan to move science results through an evaluation phase and into implementation, the Rapid Prototyping Capability fulfills a need to reduce the amount of time that has typically been required to consider the utility of new or future data streams on model outcomes. The basic functionality as shown in figure 1 consists of receiving inputs from the Solutions

of 24


Networks research results activities, performing an RPC experiment, and producing information for use in potential follow-on Integrated Systems Solutions activities.

Figure 2 depicts a functional diagram of the capabilities of a fully-functional Rapid Prototyping Capability. A typical RPC experiment consists of due-diligence systems engineering of an idea for transition from research to operations. Central to the due-diligence process, science results and partner needs are considered and environmental simulation models are selected along with existing data for baseline condition representation. Also, the RPC process identifies desired synthetic/simulated data from future sensors and/or simulation model output for enhanced/mitigated solution evaluation. A collaborative effort is followed to integrate model and data sources into the RPC for systematic evaluation. Model owners, agency experts, RPC developers and model scientists, and research evaluators collaborate to evaluate the performance of the baseline model against simulations that leverage existing NASA data sources, model derived data, and/or simulated data sets. The result of the experiment is an evaluation of whether a specified NASA science result is likely to yield an operational benefit for a partner agency and an identified pathway to a potential ISS implementation.

of 24

Figure 1. In a fully implemented state, the SN shall enable NASA science results to be harvested, allowing the selection of results to move forward for RPC evaluation. Critical to the selection of an effort for RPC evaluation are the needs to identify successful science results, baseline the conditions of the current system, identify new or future NASA data streams that might mitigate or enhance the current solution system, define the existing model schema, develop an RPC evaluation team, identify stakeholders in the solution, identify a pathway to ISS, and develop the data and model resources needed to conduct the RPC evaluation.


Figure 2. RPC Overview.

Figure 2 illustrates the RPC as a functionally implemented node in the context of a workspace at NASA Stennis Space Center with a collection of technology tools for a multidisciplinary team to perform an RPC study in a face-to-face working environment.

Although interfaces to configuring an RPC activity and evaluating outcomes are expected to occur interactively in real time, it is not expected that derivation of synthetic data, integration of new model-base capabilities, or model computational processing will occur on an interactive, real-time basis. The RPC capabilities are envisioned as supporting an evaluation workflow, some components of which will occur on a real-time or synchronous basis and some of which will involve asynchronous activities and follow-up task completion prior to proceeding to the next step in the workflow.

The capabilities of the RPC include critical infrastructure components that interface with and provide access to the NASA Enterprise Architecture tools through METIS, access to the network of NASA data products and technology resources through the “SSC RPC Gateway,” access to a variety of local and distributed modeling capabilities, and access to simulated or synthetic data sources through OSSEs, ART, and other technologies to be implemented to provide enhanced data sources for assimilation into and RPC evaluation.

of 24


2. References

2.1. Referenced documents

[1] “IEEE Guide for Information Technology --- System Definition --- Concept of Operations (ConOps) Document,” IEEE Standard 1362-1998. [2] “Extending NASA Earth-Sun System Research Results through a Systems Engineering Capacity” (Working Document), version SEC_V_5.

[3] Mississippi State University, “ Rapid Prototyping Capability for Earth-Sun System Sciences” (proposal), Principal Investigators Robert J. Moorhead and David R. Shaw.

[4] NASA Stennis Space Center, “Statement of Work: Solicitations for Proposals through the Mississippi Research Consortium (MRC) for Competitive Selection of Projects to Establish and Evolve Applied Sciences Systems Engineering Capacity in the Functional Areas of Solutions Networks, Rapid Prototyping Capability and Integrated Systems Solutions,” Sept. 9, 2005.

[5] NASA Rapid Prototyping Workshop, Hampton, Virginia, April 1920, 2006. Presentation materials.

2.2. Acronyms and Definitions

DAAC Distributed Data Active Archive CenterDSS Decision Support SystemDST Decision Support ToolsISS Integrated Systems SolutionsMRC Mississippi Research ConsortiumOSSE Operational Spacecraft Simulation ExperimentR2O Research to OperationsRPC Rapid Prototyping CapabilitySN Solutions Networks

All acronyms may be interpreted as singular or plural.

Consider the following definitions.

“Experiment” means an evaluative study for NASA conducted using the RPC. A typical study evaluates an innovative combination of data sources and scientific models.

“Data source” means any source of data used in an RPC experiment, such as Distributed Data Active Archive Center (DAAC) data products, other spacecraft observations, and simulated spacecraft observations (OSSE)

of 24


“Model” means a scientific model used in an RPC experiment. Many models provide predictive simulations of natural phenomena.

“Systems engineering” means the application of engineering principles to analysis and implementation of the transition from research to operations.

“Experiment team” means a multidisciplinary team of scientists and systems engineers that facilitates and conducts an RPC experiment.

“Steering committee” means a NASA organizational mechanism for approving and prioritizing RPC experiments.

“Technical working group” means a NASA multidisciplinary team to oversee the technical evolution of the RPC infrastructure.

3. Current situation

3.1. Background, objectives, and scope

The following is quoted from the RPC SOW [4].

NASA strategic goals and corresponding objectives are defined in “The New Age of Exploration” released by the Agency in February 2005. The guiding objectives for the NASA Earth-Sun System Division Applied Sciences Program are:

National Objective 5Study the Earth system from space and develop new space-based and related capabilities for this purpose.

NASA Objective 14Advance scientific knowledge of the Earth system through space-based observation, assimilation of new observations, and development and deployment of enabling technologies, systems, and capabilities, including those with potential to improve future operational systems.

NASA Objective 15Explore the Sun-Earth system to understand the Sun and its effects on Earth, the solar system, and the space environmental conditions that will be experienced by human explorers, and demonstrate technologies that can improve future operational systems.

The NASA Applied Sciences Program implementation strategy is aligned with these objectives. The results of NASA Earth-Sun system science include, but are not limited to the following.

NASA research spacecraft and their observations of the Earth-Sun system;

of 24


NASA models and their predictive capabilities for weather, climate and natural hazards; and

Published improvements to scientific knowledge of the Earth-Sun system.

NASA results are extended in two ways: 1) through the transition of research results to operational utilization, and 2) through projects that extend NASA research results into integrated system solutions for specific application areas of national priority (identified in the NASA Earth Science Applications Plan).

NASA has designed the system engineering approach shown in Figure 3 to applications comprising the process steps of Evaluation, Verification and Validation (V&V), and Benchmarking.

3.2. Operational policies and constraints

The current situation conforms to normal NASA organizational relationships and operations.

3.3. Description of current situation

of 24

The Verification and Validation (V&V)phase includes measuring theperformance characteristics of data,information, technologies, and/ormethods, and assessing the ability ofthese tools to meet the requirements ofthe DSS.In the Benchmarking phase, theadoption of NASA inputs within a DSSand the resulting impacts and outcomesare documented in a Benchmark Report.

Use of Systems Engineering principles leads toscalable, systemic, and sustainable solutions andprocesses, which in turn contribute to thesuccess of the mission, goals, and objectives ofeach National Application.The Evaluation phase involves understanding therequirements for and technical feasibility of Earthscience and remote sensing tools and methodsfor addressing DSS needs.

ResearchCapability

Selection

EvaluateNASA SystemComponents

as DSS Inputs

Design &ImplementIntegratedSystemSolutions

BenchmarkDSS with

NASA Inputs

Improved

Operational

System

Refine Refine Refine Refine

Evaluation V&V Benchmarking

Verify &Validate NASA

Outputs asDSS Inputs

Refine

IdentifyRequirements

&Specificationsof DSS Inputs &

Outputs

Figure 3. Systems engineering process used by the NASA Applied Sciences Program [2].


The current situation is characterized by ad hoc transition from research to operations. NASA sponsors a wide variety of projects that are aimed to facilitate the transition of specific science results to operations. Each has a different scope, covering various parts of NASA’s systems engineering approach. Each has customized immediate goals. They tend to be time-consuming (i.e. not rapid). The lack of a common infrastructure conducting such experiments tends to aggravate costs, because each experiment must find its own infrastructure.

3.4. Modes of operation for the current situation

(Not applicable.)

3.5. Users and other involved personnel

Stakeholders of the current NASA research to operations (R2O) process include the following.

Science Principal Investigators

Partner agencies

NASA Headquarters (Applied Science Program)

NASA Centers

3.6. Support environment

Not applicable currently, because of the ad hoc approach.

4. Justification for and nature of changes

4.1. Justification of changes

Figure 4 shows the importance of trying to demonstrate the utility of innovative solutions prior to initiating budget support for operational use. If a partner agency waits until a mission is flying to evaluate the utility of the science results from the mission, then it will be too late to take full advantage of the mission.

of 24


The applications community needs to begin to explore potential applications early using Operational Spacecraft Simulation Experiment (OSSE) prior to the beginning of a mission. By using this resource in conjunction with the RPC, the operations community could potentially accelerate the use of the application [2].

The RPC will significantly accelerate the evaluation of mission science results, and thereby make budget support for a mission possible much earlier than under the current situation.

4.2. Description of desired changes

Figure 5 depicts the proposed approach for transitioning NASA science results to operations. This Capabilities Document addresses only the Rapid Prototyping Capability.

Figure 5. Transitioning from research results to operations and societal benefits [2].

of 24

CrosscuttingSolutions

National Applications

Research and Analysis

ProgramApplied Sciences

Program

SocietalBenefits

EvaluationVerification

and Validation

Benchmarking

supply

demand

Rapid Prototyping

Capacity

Operations

Integrated System

Solutions

Uncertainty Analysis

Scientific Rigor

Government Agencies

&National

Organizations

NASA Earth-Sun

System Research

Solutions

Network

Operations

5-year Mission

OSSE

Budget Support

R&O demo

4-yr NRA Cycle

Figure 4. Research solicitation vs. mission cycle [2].


“Solutions Networks systematically examine the portfolio of results from NASA funded research in the seven science focus areas of the Earth-Sun System Division to find candidates that may be transitioned from research to operations or that could be integrated into solutions with specific decision support systems” [4].

“Rapid Prototyping Capability [experiments] systematically evaluate research capabilities, based on the use of specific research results in a simulated operational environment in order to evaluate components and/or configurations that could be considered for verification, validation, and benchmarking for transition from research to operations and/or into an integrated system solution” [4].

“Integrated System Solutions extend the benefits of NASA research by following a rigorous systems engineering process with our federal partners to evaluate (if necessary), verify, validate, and benchmark the assimilation of NASA research results into their decision support system(s)” [4].

4.3. Priorities among changes

(Not within the scope of this document.)

4.4. Changes considered but not included

(Not within the scope of this document.)

5. Concepts for the proposed system

5.1. Background, objectives and scope

A Rapid Prototyping Capability within NASA is needed to accelerate the transition from research to operations [3]. Operational models have strict requirements that must be verified and validated before they can become accepted by the operational partner agency. Currently, Operational Spacecraft Simulation Experiments (OSSE) are completed before any potential uses are investigated. This leads to insufficient time to investigate, verify and validate, and benchmark the science results from a sensor. By beginning rapid prototyping during the development of the OSSE, the time for transition from research to operations can be reduced. The rapid prototyping concept has been successfully applied in industry, at NOAA, and at DoD. The proposed Rapid Prototyping Capability is intended to fulfill a similar purpose in the NASA research to operations flow.

of 24


The proposed RPC infrastructure will allow multidisciplinary experiment teams, including model developers and owners, to systematically evaluate research capabilities, based on using NASA science research results in a simulated environment. Components and/or configurations that could be considered for an Integrated Systems Solution project will be evaluated. Results of NASA Earth-Sun system science include but are not limited to the following.

NASA research spacecraft and their observations of the Earth-Sun system

NASA models and their predictive capabilities for weather, climate, natural hazards, etc.

Published improvements to scientific knowledge of the Earth-Sun system

Figure 6 depicts the place of the Rapid Prototyping Capability in the context of the NASA Applied Science research-to-operations flow. One should note the wide variety of stakeholders who have an interest in this critical capability.

Figure 6. RPC problem space [5, Marley].

of 24

SN

N RPC ISS


5.2. Operational policies and constraints

An approved plan for an RPC experiment is the key input from Solutions Network. We envision the following operational process.

1. The Solutions Network activity will identify NASA science results that have good potential to add value to partner DDS. This will result in proposed RPC experiments. Other activities may also propose RPC experiments.

2. The RPC steering committee will approve and prioritize proposed RPC experiments from time to time.

3. A multidisciplinary RPC experiment team will be formed.

4. Necessary resources will be scheduled, including the RPC infrastructure, the RPC workspace, and relevant external resources.

5. The RPC experiment team will conduct the experiment.

6. Results of RPC experiment will be useful when formulating a solicitation for the follow-on Integrated Systems Solutions effort.

5.3. Description of the proposed system

The RPC shall provide an extensible architecture for defining the baseline operation of current applications, rapidly integrating new data sources, creating model-ready input data, integrating environmental simulation models, configuring model runs with typical data inputs (baseline) and new sources of data, and evaluating the performance of the environmental simulation model comparing the results of the model in its typical (baseline) configuration against results derived from the use of new data sources. The RPC shall be modular in its design and implementation and shall provide the ability to develop extensible use cases for environmental simulation models which have been integrated within the RPC framework. A suite of tools and functions will be implemented within RPC modules for managing the creation of scientific data from NASA and other data streams (SDM), manipulating scientific data to provide model ready input data through an interoperable geoprocessing engine (IGE), configuring and managing environmental simulation model runs as well as conducting or managing model computation through the model manager (MM), and providing an environment for generating performance metrics in a workbench (PMW) environment to evaluate model performance with various inputs.

of 24


NASA Stennis Space Center has designated a work space, shown in Figure 7, to host teams running RPC experiments. Although the RPC infrastructure will support virtual work groups to some extent, face-to-face meetings of interdisciplinary teams with readily available technology support will be a vital part of RPC experiments.

Figure 7. RPC room at NASA/SSC.

As noted in the RPC System Overview (section 1.4),

“The RPC capabilities are envisioned as supporting an evaluation workflow, some components of which will occur on a real-time or synchronous basis and some of which will involve asynchronous activities and follow-up task completion prior to proceeding to the next step in the workflow.”

Figure 8 provides an illustration of the typical workflow and personnel components of an envisioned RPC process that moves in stages RPC Selection to RPC Preparation to RPC Evaluation all supported by ongoing RPC Development (including evolution and maintenance) capabilities which must underpin the the RPC to support new designated evaluation. Communications and collaboration are key components that will ensure that needed models and data are successfully integrated, implemented, and RPC systems capabilities are configured to manage the evolving integrated capabilities.

of 24


of 24

Figure 8. The capabilities of the RPC are expressed in a step-wise workflow that begins with a selection process, is followed by preparation activities, and ends with an evaluation, all of which are underpinned by activities of the RPC developers.


The RPC shall comprise a collection of capabilities that will enable experiments to efficiently evaluate the usefulness of various existing or future (simulated) data streams or model derived output data streams to enhance a decision support system or simulation model. Figure 9 depicts the functional structure of the Rapid Prototyping Capability. The following section describes key modular component blocks of this diagram.

of 24

Figure 9 Functional overview of the RPC showing modular component blocks of the system and indicating interactions and interfaces between major modules.


Science Data Manager (SDM): NASA Science Data are selected for evaluation use in specific models or DSS tools. The SDM accesses the data from the NASA Earth-Sun observation sources. The sources may include the Distributed Active Archive Centers (DAAC’s), Observing System Simulation Experiments (OSSE), and other data that may be simulated through the Application Research Toolkit (ART) and similar systems. In this component of the architecture, the metadata will be developed and maintained and will be in compliance with the NASA Enterprise Architecture (EA). The SDM will supply input to the Earth Observation section of the Earth-Sun System Architecture Tool (ESAT).

Interoperable Geoprocessing Environment (IGE): The IGE will employ semantic definitions to encode processing tasks that conduct needed geoprocessing and formatting to provide model-ready input data.

Semantic Definitions:• Define geoprocessing methodology• Give meaning to the terminology of the environmental model• Remove uncertainty about the creation of information• Allow the IGE to represent instances of the concepts• Allow the IGE to produce descriptions of instances (parameters)

The IGE will comprise capabilities that include “libraries” of semantic definitions that encapsulate processing concepts, systematic encoding of model knowledge into metadata, and knowledge handling infrastructure. The fundamental capability of the IGE is to translate through geoprocessing, manipulation, and formatting the data products that measure the appropriate geophysical parameters into model-ready data inputs needed for the models being exercised. The data from the NASA sources will be processed to extract the specific information needed for the model(s) under study. The suite of atmospheric and land models, such as the NASA Land Information Systems (LIS) and the Weather Research and Forecasting (WRF) model, require a suite of initialization and forcing fields that are routinely produced by operational centers such as NCEP and ECMWF as well as by the NASA Data Assimilation activities that involve other regional and global models. The RPC will facilitate timely access to all these data sources via appropriate protocols and mechanisms. The implementation of the access and data transport mechanisms as well as the schedule and volume of data transferred will be coordinated with the operational and research agencies as necessary.

The IGE component will be a major contributor to performing experiments rapidly. A significant capability to be understood is the reusability of configured geoprocessing tasks to provide model-ready input data to a model that has been fully integrated into the RPC. It is this “reuse” capability that will enable the rapid evaluation of new data types. By associating existing geoprocessing workflows with new data types, the rapid assimilation of next-generation data into configured models should be readily achievable.

The IGE includes two sub-components; Model Data Resource Manager and Data Assimilation Tools. The Data Assimilation Tools will include transformation,

of 24


manipulation, extraction and other pre-processing tools, needed to produce the geophysical parameters input needed for the Model Manager component of the architecture. The requirements of the model and the processing steps required to produce the input will be stored in a database and used to ensure the quality of the data produced.

The Model Data Resources Manager will provide documentation and cataloging of the model requirements in a database to identify requirements that are common to multiple models, and speed development of additional prototypes that require the same input data. The database developed will supply input to the Data Products section of the NASA Enterprise Architecture tool.

Model Manager (MM): Science results and partner agency needs drive the selection of models and decision support tools to be evaluated in the RPC environment. In the RPC workflow process, the selection of a particular evaluation requires that model capabilities to perform the evaluation are available and configured for the desired data stream to be considered in the evaluation. The selection task should trigger the following critical capabilities questions:

1) Is the desired model or decision support tool integrated into the RPC model manager along with the necessary “model knowledgebase?”

2) Are the data desired for evaluation configured for ingest and assimilation within the RPC?

3) Are the needed geoprocessing tasks to create model ready data from desired evaluation input data streams fully implemented?

The MM component of the proposed architecture also has two sub-components that correspond with the Knowledge Base: the Model Base Manager and Model Results Resource Manager. These components correspond to the Model and Analysis Systems and Model Outputs/Predictions parts of the Knowledge Base. The Model Manager component will be coordinated with the Earth System Modeling Framework (ESMF) to contribute to the building of an infrastructure designed to increase the performance, portability, interoperability, and coordination in the modeling community.

Performance Metrics Workbench (PMW): Environmental simulation models, decision support tools, and custom data product outputs are systematically stored and tested against baseline outputs and outputs created from contrasting test configurations. Performance parameters are computed and examined utilizing visualization methods as well as tabular, graphical, and statistical measures. The PMW will provide tools for assessing whether there is significant improvement in a model’s performance. The results of the model using NASA data will be compared to the current results and the difference evaluated. A key component of the PMW will be the visualization of the model results. The Model Scenario Comparison Toolkit will provide capabilities that include quantitative tools to evaluate model performance. These tools include the statistical analysis of the numerical results to determine the significance of any differences in outcome.

of 24


5.4. Modes of operation

The RPC as a whole will have a single mode of operation, customized for support the experiment at hand.

On a technical level, each tool will have its own mode of operation.

5.5. User classes and other involved personnel

Stakeholders of the proposed Rapid Prototyping Capability include the stakeholders of the current NASA applied science systems engineering process, plus the organizations involved in implementation and support of the RPC.

Science Principal Investigators

Partner agencies

NASA Headquarters (Applied Science Program)

NASA Centers

NASA Stennis Space Center

Mississippi Research Consortium team

Mississippi State University

University of Mississippi

Science Systems and Applications Inc. (SSAI)

Institute for Technology Development (ITD)

5.6. Support environment

To be determined. Contracts have not been awarded for the period after December 31, 2007.

6. Operational scenarios

This section provides scenarios that show how the Rapid Prototyping Capability could be used to perform experiments that will facilitate the transition of NASA science results to operational benefits through partner agencies. Each scenario is a use case.

of 24


6.1. Scenario for idealized experiment evaluating data sources

Use Case: Perform experiment using multiple data sources and a single model--------------------------------------------------CHARACTERISTIC INFORMATIONGoal in Context: Evaluate whether a specified NASA data source, compared to a baseline data source, can add value to a specified numerical modelScope: This use case includes activities by people, processing by computational resources, and communications. Level: Top levelPreconditions:

1. Complete RPC experiment plan2. Formation of RPC systems engineering team for this

experiment, including data and modeling specialists3. Access to appropriate external resources (e.g. data, models,

host computers)4. Availability of RPC node(s) for needed timeframe

Success End Condition: Completed RPC evaluation reportFailed End Condition: Experiment not performedPrimary Actor: RPC systems engineering team, Trigger: Management approval and schedule----------------------------------------MAIN SUCCESS SCENARIO1. Verify access to appropriate external resources.2. Acquire baseline data set.3. Acquire or produce experimental data set.4. Preprocess baseline data set, if necessary, for compatibility with the model. (e.g.

transformation, manipulation, extraction, scaling, interpolation, reformatting)5. Preprocess experimental data set for compatibility with the model.6. Run the model for the baseline data set.

6.1. Configure the model for the baseline data set.6.2. Run the model.6.3. Acquire baseline model run results.

7. Run the model for the experimental data set.7.1. Configure the model for the experimental data set.7.2. Run the model.7.3. Acquire experimental model run results.

8. Compare results of model runs.9. Analyze the uncertainty of modeling results.10. Evaluate potential value of experimental data source to target DSS.11. Publish scientific results of the experiment in a peer-reviewed venue.12. Submit results to the ESAT developers.----------------------EXTENSIONSSome experiments will have a more complex configuration of experimental conditions.--------------------

of 24


SUB-VARIATIONS(none)----------------------RELATED INFORMATION (optional)Priority: This scenario is a core RPC function.Performance Target: a few weeks to a few months depending on experiment detailsFrequency: One scenario per experimentSuperordinate Use Case: (none)Subordinate Use Cases: (none)Channel to primary actor: RPC node facilitiesSecondary Actors: (specified by RPC experiment plan)Channel to Secondary Actors: (specified by RPC experiment plan)----------------------------

6.2. Scenario for idealized experiment evaluating models

Use Case: Perform experiment using baseline data sources and multiple models--------------------------------------------------CHARACTERISTIC INFORMATIONGoal in Context: Evaluate whether a specified model or model configuration, compared to a baseline model, using baseline data sources, can add value to a specified decision support application.Scope: This use case includes activities by people, processing by computational resources, and communications. Level: Top levelPreconditions:

1. Complete RPC experiment plan2. Formation of RPC systems engineering team for this

experiment, including data and modeling specialists3. Access to appropriate external resources (e.g. data, models,

host computers)4. Availability of RPC node(s) for need timeframe

Success End Condition: Completed RPC evaluation reportFailed End Condition: Experiment not performedPrimary Actor: RPC systems engineering team, Trigger: Management approval and schedule----------------------------------------MAIN SUCCESS SCENARIO1. Verify access to appropriate external resources.2. Acquire baseline data sets.3. Preprocess baseline data set, if necessary, for compatibility with the baseline model.

(e.g. transformation, manipulation, extraction, scaling, interpolation, reformatting)4. Preprocess baseline data set for compatibility with the experimental model.5. Run the baseline model for the baseline data set.

of 24


5.1. Configure the baseline model.5.2. Run the model.5.3. Acquire baseline model run results.

6. Run the experimental model using the baseline data set.6.1. Configure the experimental model for the baseline data set.6.2. Run the model.6.3. Acquire experimental model run results.

7. Compare results of model runs.8. Analyze the uncertainty of modeling results.9. Evaluate potential value of experimental model to target DSS.10. Publish scientific results of the experiment in a peer-reviewed venue.11. Submit results to the ESAT developers.----------------------EXTENSIONSSome experiments will have a more complex configuration of experimental conditions.--------------------SUB-VARIATIONS(none)----------------------RELATED INFORMATION (optional)Priority: This scenario is a core RPC function.PerformanceTarget: a few weeks to a few months depending on experiment detailsFrequency: One scenario per experimentSuperordinate Use Case: (none)Subordinate Use Cases: (none)Channel to primary actor: RPC node facilitiesSecondary Actors: (specified by RPC experiment plan)Channel to Secondary Actors: (specified by RPC experiment plan)----------------------------

7. Summary of impacts

7.1. Operational impacts

The Rapid Prototyping Capability will be instrumental in the shift from the current ad hoc approach to a systems engineering to transitioning research to operations.

7.2. Organizational impacts

The following are impacts on organizational issues that will need resolution by the time the RPC infrastructure has been developed (January 2008).

NASA will need to establish an organizational mechanism for approving and prioritizing RPC experiments. In this document, we call such an organization a

of 24


“steering committee.” An organizational mechanism for balancing the interests of multiple stakeholders will be essential to the long-term success of RPC.

Each RPC experiment will need a temporary multidisciplinary team to facilitate and conduct the experiment. In this document, we call such an organization an “experiment team.” Each team will exist only for the duration of the experiment.

The RPC infrastructure will need a multidisciplinary team to oversee the technical evolution of the infrastructure. In this document, we call such an organization a “technical working group.”

7.3. Impacts during development

The experiments that will be conducted during development of the RPC (January 2006–December 2007) will, in their own right, be useful to NASA for facilitating the transition of research to operations.

The development of RPC (January 2006–December 2007) will not interfere with NASA’s current research-to-operations transition efforts.

8. Analysis of the proposed system

8.1. Summary of improvements

The proposed Rapid Prototyping Capability is targeted to provide the following improvements.

The RPC infrastructure will provide a suite of tools that will be useful for rapid conduct of RPC experiments across a range of observational systems and science models. Such a coordinated set of tools is not currently readily accessible to a RPC experiment team.

The RPC facility at NASA Stennis Space Center will provide a place for an RPC experiment team to work face-to-face, and thus, expedite progress on the experiment. Such a facility is currently not commonly used by ad hoc experiments.

The RPC experiments conducted during development (January 2006–December 2007) will illustrate the long-term potential for rapid effective RPC experiments.

of 24


8.2. Disadvantages and limitations

The proposed Rapid Prototyping Capability has the following limitations.

Although simulations of future planned data streams is time consuming and expensive, one of the primary drivers for the RPC is the need to understand the usefulness of next-generation data products. Therefore, a central aspect of the RPC is the use of simulated or synthetic data streams to employ in applications for comparison against baseline outputs. In-so-far as is possible, existing tools that enable the rapid simulation of next-generation data streams will be employed to arrive at “rapid” answers to the usefulness questions that surround next-generation satellite sensor data streams and derived scientific data products.

Substantial programming of software, such as major modification and recoding of existing models to make them RPC compliant, is outside the scope of RPC experiments, because such programming efforts will defeat the goal of “rapid” results.

RPC experiments will perform due-diligence systems engineering of limited prototypes for the purpose of providing evidence of the practical value of NASA science results. In contrast, the Integrated Systems Solutions projects will address engineering issues relevant to verification and validation of the solution to fully meet the needs of partner agencies.

The proposed system (January 2008) will consist of two computational nodes, one at NASA Stennis Space Center and one at Mississippi State University. The infrastructure will be designed for straightforward replication of nodes at additional networked sites.

The proposed system (January 2008) will have one physical meeting facility at NASA Stennis Space Center. The design will allow straightforward replication at other locations.

of 24

concept of operations template - gri.msstate.edu · web viewthe format of this document follows...

Documents