towards a virtual solar- terrestrial observatory · the virtual solar-terrestrial observatory...
TRANSCRIPT
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Towards a Virtual Solar-Terrestrial Observatory
Peter Fox ([email protected])HAO/NCAR
(with Jose Garcia, Patrick West, Don Middleton and Stan Solomon)Work partially funded by NSF
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Outline� Concept and user needs� What’s new about a VSTO?� Integration with existing systems� Functionality� Technology� Building on experience� Status
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Concept and user needsGoal - find the right balance of data/model
holdings, portals and client software that aresearchers can use without effort orinterference as if all the materials wereavailable on his/her local computer.
The Virtual Solar-Terrestrial Observatory (VSTO) is proposed to be:• a distributed, scalable education and research environment for
searching, integrating, and analyzing observational, experimental andmodel databases in the fields of solar, solar-terrestrial and spacephysics
VSTO would comprise:• a system which provides virtual access to specific data, model, tool
and material archives containing items from a variety of space- andground-based instruments and experiments, as well as individual andcommunity modeling and software efforts bridging research andeducational use
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Concept and user needsIn discussions with data providers and users, the needs are clear:
``Fast access to `portable' data, in a way that workswith the tools we have; information must be easyto access, retrieve and work with.'’
Too often users (and data providers) have to deal with theorganizational structure of the data sets which variessignificantly --- data may be stored at one site in a small numberof large files while similar data may be stored at another site in alarge number of relatively smaller files. There is an equally largeproblem with the range of metadata descriptions for the data.Users often only want subsets of the data and struggle withgetting it efficiently. One user expresses it as:
``(Please) solve the interface problem.''
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
What’s new in the VSTO?
• Datasets alone are not sufficient to build a virtualobservatory
• VSTO must address the interface problem to bringdata to the users' tools, and to the tools within theVSTO, effectively and scalably
• VSTO will leverage the development of schema (e.gEarth System Grid, VSO/VHO) that adequatelydescribe the syntax (name of a variable, its type,dimensions, etc. or the procedure name andargument list, etc.) and semantics (what the variablephysically is, its units, etc. or what the proceduredoes and returns, etc.) of the datasets and tools.
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
What’s new in the VSTO?• A Grid-enabled (http://www.globus.org) virtual
observatory minimizes the time to make dataavailable and usable. Data does not have to bemoved or reformatted, only registered with thecatalog. It is then available from the VSTO webportal or the user's preferred application which hasaccess to the VSTO interfaces
• VSTO will address the interdisciplinary metadataand ontology problem - bridging terminology and useof data across disciplines
• Aims to integrate tools, models, and data• A basis for a framework for building and distributing
advanced data assimilation tools
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Integration: MLSO
Near real-timedata fromHawaii from avariety of solarinstruments, asa valuablesource forspace weather,solar variabilityand basic solarphysics
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Integration: CEDARWEB
Communitydata archive,documents,andsupport.
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Integration: VSO query schema
Forms abasis toorganizethecataloginterfacesfor alltypes ofdata,images,etc.
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Functionality
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Technology: Community Data Portal
Free text search
Applications
Live Access
News
Authentication
THREDDS catalog
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Technology: Community Data Portal
THREDDScatalogsare formedinto alogicalhierarchywithsimpletext
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Technology: Community Data Portal
Access tometadata ofvarious levels,e.g. DublinCore (shown) inan easy to readand retrieveformat
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Technology: Earth System GridDOE/SciDACproject toaddress a verysimilar needin the climatemodelingcommunity;one discipline,model data,one format,many accessinterfaces.
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Experience: CEDARWEB
Data query,selection andretrievalinterface,withintegratedtools, e.g.ability to plot(preview) databeforeretrieving it.
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Experience: CEDARWEB
Ability to quickly plotdata to assesssuitability, quality,and produce a quickcopy with somecustomization for apreliminary study.
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Experience: CEDARWEBCEDARWEB has been developed and improved overmore than 10 years of interaction with users, dataproviders, and a community steering committee. Eachof these elements has directly contributed to changes inwhat services are provided, what information andmaterials are made available via the web site and whatlevels of authorization and authentication are required.
Biggest lesson: we have plenty of technology, so getclose to the people and listen to them to buildsomething that is useful and that will have the longevitywe all seek.
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Experience: CEDARWEBDon’t justprovide data,but also buildin communityinformationand ancillaryinformationthat is ofvalue.
CEDAR Workshop 2004 - Towards an Integrated Data Environment, July 2, 2004
Status� Seeking community involvement in:
– Steering committee– User working group– Development of detailed requirements
� Collaborating with ESG to leverage efforts withinthat project
� Collaborating with computer scientists todevelop the ontologies includinginterdisciplinary dictionaries
� Collaborating on efforts in data assimilation toguide the development of data/model interfaces
� Please make contact ([email protected]) for moreinformation