testing - an essential aspect of establishing an...
TRANSCRIPT
Testing - an essential aspect of
establishing an SDI
Clemens Portele, Anders Östman, Michael Koutroumpas, Xin He, Janne Kovanen, Markus Schneider, AndrianiSkopeliti
INSPIRE Conference 2011
30 June 2011
Reminder
RegistersService
Metadata
Data Set
Metadata
Registry Service Discovery Service
Metadata
Services
Access Control, Authentication
Application and Geoportals
Internet
InvokeSD
Service
Transf.
Service
Spatial Data Set
Thematic DS
Framework for
harmonized DS
Spatial Data
View
Service
Download
Service
INSPIRE Services
• Based on OGC Services (WMS, WFS, WPS, etc)
• Quality-of-Service requirements (performance, capacity,
availability)
• Multilingual support
• Additional service metadata
Reminder
RegistersService
Metadata
Data Set
Metadata
Registry Service Discovery Service
Metadata
Services
Access Control, Authentication
Application and Geoportals
Internet
InvokeSD
Service
Transf.
Service
Spatial Data Set
Thematic DS
Framework for
harmonized DS
Spatial Data
View
Service
Download
Service
Spatial data sets in INSPIRE
• Data specifications based on ISO 19100 series
• Using GML for encoding
• Specifying additional requirements
• Specifying layers in View Services
ESDIN
• An eContentplus Best Practice Network project
• September 2008 to February 2011
• Coordinated by EuroGeographics
• Key goal: help National Mapping and Cadastral Agencies (NMCAs) member states, candidate countries and EFTA States prepare their data for INSPIRE Annex I spatial data themes and improve access
ESDIN
• In total 7 ESDIN-related presentations at INSPIRE 2011
• In particular:
• Thu 16:00, David Overton, ESDIN - the geospatial reference data and services for INSPIRE
• Fri 09:20, Antti Jakobsson, European Location Framework- How to provide geospatial reference data and services for Europe based on INSPIRE
• This talk focuses on the ESDIN test environment
ESDIN specifications
• Use INSPIRE service specifications
• Extend INSPIRE data specifications
– Additional information that already is part of current data
products of EuroGeographics and which is used by
existing customers has been added
– Information that may be void in the INSPIRE data
specifications (i.e. is not necessarily provided in INSPIRE),
but which can be provided by NMCAs is made mandatory
– Minimum data quality levels have been specified for
European NMCA data
ESDIN testing
• Provide a systematic approach to testing the technical results
of the ESDIN project and support implementation activities
• General testing approach:
– Automated testing
– Black box testing
OGC compliance testing (CITE)
OGC CITE tests vs
ESDIN test requirements
• OGC CITE tests
– Use a test data set
– Test a software product
– Not for all service and data standards available
• ESDIN Test requirements
– Test a service instance, not the capabilities of a software
product
• Products used in ESDIN have to pass CITE tests where they are
available
– Test against INSPIRE and ESDIN extensions
Publication
data
Download
Service
Transformations
ESDIN data production
and publication workflow
Source to
target model
Source
data
Coordinates
Generalise
Translate
Other data
Edge-
matching
View Service
Authentication
Service
Test targets – overview
• Publication of spatial data in the SDI
– INSPIRE view services / OGC WMS
– INSPIRE download services / OGC WFS
– INSPIRE view/download services via SAML/Shibboleth-
based authentication services
– INSPIRE coordinate transformation services
– INSPIRE and ExM data sets
• Data production processes
– Medium-to-small-scale generalisation
– Data quality evaluation
– Edge-matching processes
Test cases – overview
• Test cases documented using a
common template for all test
targets
• In total 165 pages of test cases
Publication
data
Download
Service
Transformations
ESDIN data publication test target (example)
View services (WMS)
Source to
target model
Source
data
Coordinates
Generalise
Translate
Other data
Edge-
matching
View Service
Authentication
Service
Publication
data
Download
Service
Transformations
ESDIN data publication test target (example)
View services (WMS)
Source to
target model
Source
data
Coordinates
Generalise
Translate
Other data
Edge-
matching
View Service
Authentication
Service
INSPIRE view service test cases
• Service metadata validation
• Support for INSPIRE service metadata extensions
• Performance tests
• Availability tests
• Capacity tests
• Scaling tests
• Variations of valid and invalid requests
• Support for INSPIRE layers
Publication
data
Download
Service
Transformations
ESDIN data publication test target (example)
INSPIRE and ExM data sets
Source to
target model
Source
data
Coordinates
Generalise
Translate
Other data
Edge-
matching
View Service
Authentication
Service
Publication
data
Download
Service
Transformations
ESDIN data publication test target (example)
INSPIRE and ExM data sets
Source to
target model
Source
data
Coordinates
Generalise
Translate
Other data
Edge-
matching
View Service
Authentication
Service
INSPIRE data set test cases
• Basic validation
• against XML Schema
• against OCL constraints
• Spatial and temporal information
• Coordinates in ETRS89 and in Europe
• Time zone information provided
• Links to other resources
• Code list encoding
• Encoding of references to other features
• Locale provided for localised character strings
• Other tests
• Encoding of nil values
• Identifier provided
• Correct feature collection container
© 2010 Open Geospatial Consortium 17
Using soapUI and loadUI
(loadUI for load tests)
Modified version of soapUI to support
additional capabilities for implementing
test cases
Experiences in ESDIN• Very positive feedback from National Mapping and Cadastral Agencies within the
project transforming the data and setting up the ESDIN web services
– Tests enable the discovery of problems
• by the developer
• at an early stage of the implementation
– Automated tests allow monitoring the operation, too
• Positive feedback from EU project review: „ESDIN is one of the first projects that
also provide appropriate testing tools that can check the conformity to the ESDIN
data specifications as well as conformity to INSPIRE. These results should be
forwarded to the INSPIRE community.“ � one of the reasons we are here
• Additional work needed
– Improve reporting of reasons for failed tests
– Align with progress in INSPIRE specifications (new Technical Guidance
documents for network services, new data specifications, etc.)
Additional information
• Test framework:http://services.interactive-
instruments.de/download/ETF.zip
• Deliverables:http://www.esdin.eu/project/summary-esdin-project-public-
deliverables
– D12.4 Test cases
– D12.5 Test components
– D12.6 Test results
Final remarks
• The ESDIN Test Framework is available under BSD license
• The ESDIN consortium is preparing the steps to maintain and
improve the project results in the future and the test
components will be part of this
– Currently the test framework has its home in a Subversion
server at interactive instruments, but we plan to move this
as part of this process
– We are looking for opportunities for further collaboration to
improve the test framework and make it available for future
use in INSPIRE and national/community SDIs