wpx - dn - smartopendata · sites. upon discussion with domain experts, mapping between individual...

This project has received funding from the European Union’s Seventh Programme for research, technological development and demonstration under grant agreement No 603824.

Visualization Framework Deliverable D4.2: Public

Keywords: OGC, RDF, Geo-Spatial, SPARQL, Visualization

Linked Open Data for environment protection in Smart Regions

D4.2 Visualization Framework SmartOpenData project (Grant no.: 603824)

Version 1.0 of 32 © SmartOpenData Consortium 2014

Table of Contents

Table of Contents ....................................................................................................................... 2

List of Figures .............................................................................................................................. 3

List of Tables ............................................................................................................................... 4

Executive Summary .................................................................................................................... 6

1 Uses cases ........................................................................................................................... 7

1.1 Representation of parcels dataset by using GeoServer ............................................. 7

1.1.1 Parcels data scheme ............................................................................................... 8

1.1.2 GeoServer and Openlayers .................................................................................... 9

1.1.3 Geo-filtering: ECQL ............................................................................................... 10

1.2 Representation of parcels dataset by using the SEFARAD map widget................... 11

1.2.1 Protected sites dataset......................................................................................... 11

1.2.2 SPARQL Query and results data ........................................................................... 16

1.2.3 Faceted search ..................................................................................................... 18

1.2.4 Showing the data: results table ............................................................................ 19

1.2.5 OpenStreet Map: GeoJSON representation ......................................................... 20

2 Conclusion and future works ........................................................................................... 22

2.1 Project outcomes ..................................................................................................... 22

2.2 Future Works ............................................................................................................ 23

3 Installation Instructions .................................................................................................... 24

3.1 Quick Install .............................................................................................................. 24

3.2 Detailed Installation Instructions ............................................................................. 25

4 User manual ..................................................................................................................... 28

4.1 Add widgets .............................................................................................................. 28

4.2 Faceted search ......................................................................................................... 29

4.3 SPARQL editor .......................................................................................................... 29

5 Developer manual ............................................................................................................ 31



List of Figures

Figure 21 Feature Types Details. ................................................................................................ 9

Figure 22 Openlayers map. ...................................................................................................... 10

Figure 23 SmartOpenData layout. ............................................................................................ 10

Figure 24 Protected sites tables and view. .............................................................................. 12

Figure 25 Linear layout. ............................................................................................................ 18

Figure 26 Accordion layout. ..................................................................................................... 19

Figure 27 Results Widget with protected areas. ...................................................................... 19

Figure 28 Protected sites map. ................................................................................................ 21

Figure 29 An overview of SmartOpenData app in the SEFARAD tool. ..................................... 24

Figure 30 An overview of SmartOpenData app in SEFARAD tool ............................................ 27

Figure 31 Available widget templates in SEFARAD .................................................................. 28

Figure 32 Enabling faceted browsing feature for geo-spatial data. ......................................... 29

Figure 33 SPARQL editor .......................................................................................................... 30



List of Tables

Table 13 Dataset scheme. .......................................................................................................... 8

Table 14 Data INSPIRE designation. ......................................................................................... 15

Table 15 Sefarad requirement list. ........................................................................................... 25



Contractual Date of Delivery to the EC: May 2015

Actual Date of Delivery to the EC:

Editor(s): UPM

Contributor(s): Carlos Ángel Iglesias, Geovanny Poveda (UPM), John O’Flaherty (MAC), Giovanni Tummarello (Sindice), Karel Charvat (HSRS), Dumitru Roman (SINTEF), Pēteris Brūns (IMCS), Martin Tuchyna (SAŽP), Maria José Lucena e Vale (DGT), Azucena Sierra (TRAGSA)

DOCUMENT HISTORY

Version Version date Responsible Description

0.0.1 10 December 2014 UPM Initial draft, TOC, call for contributions from others.

0.0.2 23 April 2015 DGT Merged with contributions

0.0.3 14 May 2015 UPM Released version, minor corrections still required

1.0 22 May 2015 TRAGSA Final revision

The information and views set out in this publication are those of the author(s) and do not necessarily reflect the official opinion of the European Communities. Neither the European Union institutions and bodies nor any person acting on their behalf may be held responsible for the use which may be made of the information contained therein.

Copyright © 2014, SmartOpenData Consortium.



Executive Summary

This deliverable describes the design, architecture and use of a semantic visualization tool for representing linked geospatial data for the SmartOpenData (SmOD) project. A semantic visualization linked data tool named SEFARAD has been proposed. This tool provides mechanisms for enabling non skilled GIS users to explore and visualize different kinds of linked data resources with a high level description of classes, properties and relations. SEFARAD enables GIS users to apply faceted browsing features to go beyond the data exploration process to apply specialized linked data visualization features. Furthermore, SEFARAD provides a utility for creating and processing human-readable SPARQL queries as a part of the exploration and visualization process. Functionalities provided by the SEFARAD has been evaluated by using geospatial data from Slovak Environment Agency (SAZP). In this evaluation, about 42770 RDF descriptions that contain information about protected parcels in Slovakia, have been processed, analysed and represented.

http://www.smartopendata.eu/



1 Uses cases

For evaluating the functionalities provided by SEFARAD, two uses cases have been proposed. In the first use case, a GEO map server utility has been used for representing geospatial information related to protected parcels in Spain. In the second use case, a widget map has been used for representing geospatial data from the Slovak Environment Agency (SEA), in particular protected parcels in Slovakia.

1.1 Representation of parcels dataset by using GeoServer

In this section we will describe the use of shapefiles dataset provided by Tragsa1 for the SmartOpenData project. The main purpose of this case study is to test Sefarad performance with GeoServer. Tragsa has provided us a new dataset about different kinds of parcels in Spain in DBF and Shapefile format. The data scheme is shown in table 14.

1 http://www.tragsa.es/



Table 1 Dataset scheme.

1.1.1 Parcels data scheme

We have stored all the shapefiles in our local GeoServer installation. From there, we are able to see all the different parcels, see its feature types, preview them with Openlayers directly from GeoServer, test ECQL filtering, etcetera. For this case study, we have chosen the td_0307_sigpac parcels, whose feature types are detailed in Figure 29.



Figure 1 Feature Types Details.

1.1.2 GeoServer and Openlayers

In this case, we do not need the GeoProxy to convert the retrieved data. We can query to GeoServer directly from Sefarad and represent the geographical data in an Openlayers map. For this case, the SPARQL query is shown in listing 17.

Listing 1 SMOData Dataset SPARQL query.



We must also indicate the graph of the requested data, which is: http://dataset1.smartopen.gsi.edu. After receiving the results, we can directly represent them without any conversion in an Openlayers map, as shown in Figure 30.

Figure 2 Openlayers map.

1.1.3 Geo-filtering: ECQL

As we can see in Figure 30, this widget includes a filtering box which allows us to test geo-filtering with the retrieved data. We can include any ECQL query in this field and the geo-filtering module will automatically update and rerun the query to receive the information that meets the new criteria. Figure 31 shows the complete layout of this case study.

Figure 3 SmartOpenData layout.



1.2 Representation of parcels dataset by using the SEFARAD map widget

In this section we will describe a case study that will help us to try the main features of Sefarad, including all those relative to query a SPARQL endpoint, manage the data with different filtering methods and display the final results in some graphical and geographical widgets. In this case study we will work with a dataset provided by the Slovak Environmental Agency (SEA2) which contains information about harmonised protected sites in Slovakia. The dataset is based on the INSPIRE3 Data Specification. This case study involves the following activities:

Query a Fuseki dataset.

Edit and run our own SPARQL query.

Display the retrieved data in a graphical table.

Point the geographical position of the results into an Open Street Map3 by using Openalayers.js framework.

Add some filtering widgets to test faceted search and keyword search.

Display the accordion layout to include many facets for filtering.

Display GeoJSON polygons in an OpenLayers map.

Edit and run some SPARQL queries in the dashboard tab for managing some static values.

1.2.1 Protected sites dataset

SEA provided source files for Geoserver workspaces related to Slovak protected sites feature type. Source files can be downloaded from SEA website4.

Protected sites data are stored in GIS database as simple features in several database tables. Tables slightly differ in structure; they contain some common and some different fields. The structure of database tables is application dependant and was designed in the past during applications development; different applications are used for different kind of protected sites. Upon discussion with domain experts, mapping between individual protected sites tables and protected sites designation schemas was established. Thus, unifying view could be constructed, joining several protected sites tables into one logical database view with common set of data fields. Set of common data fields were chosen according to INSPIRE Protected Site simple profile application scheme. Fields for designation scheme and designation scheme value (both mandatory in application schema) were added to view for easing later mapping. Figure 24 shows the data scheme.

2 http://www.sazp.sk/

3 http://www.openstreetmap.org/

4 http://inspire.geop.sazp.sk/geoserver/www/eenvplus/ps harmonisation.tgz



Figure 4 Protected sites tables and view.

The final data have the following fields and information values, which will be used to test faceted browsing and geo filtering with ECQL5.

5 http://docs.geoserver.org/latest/en/user/_lter/ecql reference.html



Table 2 Data INSPIRE designation.



1.2.2 SPARQL Query and results data

In this case, we store the information into a local Fuseki server and we query it from Sefarad. Query for retrieving the corresponding information is shown below.

Listing 2 Slovakian Demo SPARQL.

At this point, a piece of the information returned from the server is as follows.



Listing 3 Slovakia results JSON example.

1.2.3 Faceted search

In this case we have a greater number of facets, allowing us to observe the results when combining it. After adding a Tag Cloud widget for each of the facets we want to filter by, we have the next application layout.

Figure 5 Linear layout.

As we can see in the image above, when the number of facets becomes higher and higher, it becomes an impractical interface, because we cannot show all the widgets in the window at the same time, and if we filter by a facet whose widget is lower, we must 'remove' the display map, so we cannot see how the results are updated. To solve this problem, we developed another type of layout: the accordion layout [4.7.2.2]. Applying respective layout, the previous screen becomes as follows.



Figure 6 Accordion layout.

1.2.4 Showing the data: results table

The main widget available in Sefarad for exploring and ordering the final results is the Results Table widget. You can add it from the 'Add new widget' section. Once you have done it, the widget automatically recognises the different facets of the results and draws them into a table like the one above (Figure 27).

Figure 7 Results Widget with protected areas.

As we work with large amounts of data, we realized the need to do this widget as much configurable as possible, so that the user can select how many items to display per section, which columns to show or hide, etcetera. Furthermore, we included in the results table a search box that the user can use to search within the filtered results, which is an extra



functionality apart from filtering technologies. All those configuration options can be shown in the figure above.

1.2.5 OpenStreet Map: GeoJSON representation

In this case study, we will use the features for polygons representation that Openlayers offers. First of all, we need to convert the retrieved geoSPARQL data to GeoJSON data by using the GeoProxy. After doing that, we will have a FeatureCollection. This list will contain one feature per parcel in the dataset. An example of one of these features is shown in the next listing.

Listing 4 Slovakian feature example.

Drawing it with Openlayers and OpenStreet Map, we get the following map result.



Figure 8 Protected sites map.



2 Conclusion and future works

2.1 Project outcomes

The main outcome of this project is the developed system. By extending the existing Sefarad framework with the suitable modules and widgets, we have created a powerful querying and visualization framework for Geo Linked Data, enabling final user to query any dataset that users want and manage, index, sort and filter the retrieved data.

Due to integration with different types of databases as data sources, as explained in Section 4.5, the user is able to store data in many formats and query, filter and represent it from Sefarad. Furthermore, the user can query any semantic endpoint due to the compatibility with Semantic Web and Linked Data specifications. Moreover, the GeoProxy (Section 4.4) solves format compatibility issues for representation with Openlayers. Otherwise, the development of the User Management Module, detailed in Section 4.8, provides multiple important features. On the one hand, this module allows the users to save and load their own preferences and settings each time they use Sefarad, which makes the application more configurable and customizable. In the other hand, the application administrator can use this module to manage the information (credentials, permissions, etcetera) about the different users. Moreover, the integration with MongoDB can be exploited to store large amounts of data in JSON format thereby improving application performance. The geo filtering capabilities included in the tool, besides to faceted browsing and keyword search (Section 4.6), enable new powerful filtering options for geographic data. By using CQL language, the user can write filtering queries with a familiar text-based syntax, which is thus more readable and better-suited for manual authoring. To make the application more responsive and appropriate to filter data with many fields of information, we have developed the accordion layout described in Section 4.7.2.2. This provides the user a cleaner and more usable interface, making it easier to display filtering and representation widgets on the same screen. Finally, the inclusion of the Grunt task runner (Section 4.9.2) and the reorganization of the project source files facilitate the work of developers. Now, any developer can create a new widget just by using the widgets template and running the corresponding Grunt command, as explained in Appendix B.

All these features have been tested in the different case studies (Section 5). These experiments have allowed us to verify the correct operation of the system and all its components and include some improvements needed for a better user experience.



2.2 Future Works

The project outcome can also serve as a solid base for future work and development. In the following points some fields of study or improvement are presented to continue the development.

Enable Sefarad to handle heterogeneous data collections, so that different queries can be performed, storing multiple data types and handle them separately or combined.

Improve the performance of the application when large amounts of data are queried, so the service does not get blocked or offers an excessively long time to respond.

Integrate SirenDB as a local database, in order to improve the performance of the application with large amounts of data and take advantage of SOLR indexing.

Improve geographical search operators, such as intersections between polygons or polygons contained within other polygons.

Integrate Quepy3 Framework, to allow non-technical users to execute queries in natural language.

Improve the installer. It has been developed a first version of the installer, but it should be improved so that it installs everything needed for the execution of the application, such as MongoDB, Grunt.js, etc.



3 Installation Instructions

This section explains how users can install, configure and deploy the SEFARAD tool. Two options are currently available; quick install and detailed installation.

3.1 Quick Install

We provide a preconfigured package for the SmartOpenData project. This version includes a standalone and light-weight bundle package to publish geo-linked data. It is meant as a tool for rapid prototyping and deployment of a linked geo spatial data for the SmartOpenData project. Please follow the steps described below:

1. Download or clone the SmartOpenData master package from our github repository (https://github.com/gsi-upm/demo-smartopendata-bundled-package/) and uncompress it.

2. Run the install script by typing the commands listed below:

%shell%> cd demo-smartopendata-bundled-package/bin

%shell%> ./allinone.sh --config ../allinonebundle.conf

Listing 5 Command for deploying the SmartOpenData bundle package.

3. To check if the SmartOpenData app was successfully deployed, access the URL listed below:

http:/localhost/index.html#/sparql/slovakiaPolygonsDemo

Listing 6 URL for accessing the deployed SEFARAD APP.

Now, you should be on the webpage shown in figure 29.

Figure 9 An overview of SmartOpenData app in the SEFARAD tool.

https://github.com/gsi-upm/demo-smartopendata/



3.2 Detailed Installation Instructions

We also provide the source package for the SmartOpenData project. This version includes the SEFARAD code application as well as its libraries. Before following this guide, make sure you meet the requirements listed in table 15. This document should not describe the installation and configuration process of each of them. However, a list of links is provided for explaining how users can install and configure them.

Requirement Url for installation instructions

1 PHP http://php.net/manual/en/install.php

2 Mongo DB http://docs.mongodb.org/manual/installation/

3 Grunt http://gruntjs.com/installing-grunt

4 Apache http://httpd.apache.org/docs/2.2/install.html

5 Java https://docs.oracle.com/javase/7/docs/webnotes/install/

6 Fuseki http://jena.apache.org/documentation/serving_data/

Table 3 Sefarad requirement list.

1. DOWNLOAD THE SEFARAD PACKAGE

Download or clone the SEFARAD master package from our github repository (https://github.com/gsi-upm/demo-smartopendata/). Uncompress and move it to {$path}.

Where:

$path corresponds to apache web directory.

2. FUSEKI CONFIGURATION

Make the following changes to the config.ttl file located in {$pathfuseki}

$pathfuseki corresponds to the fuseki directory of your system.

1. Edit the service section by adding a description for the linked data resource to be included. Listing 18 shows the values to be included.

<#serviceName> rdf:type fuseki:Service ; fuseki:name "SmartOpenData demo" fuseki:serviceQuery "query" ; fuseki:serviceReadGraphStore "get" ; fuseki:dataset <#smartopendata> ;

Listing 7 Description for a Fuseki service.

2. Edit the configuration section by specifying the location of the RDF file to be included. A RDF file named sod.rdf is provided in the data folder of the SEFARAD package. Listing 19 shows the values to be included.

http://php.net/manual/en/install.php

http://docs.mongodb.org/manual/installation/

http://gruntjs.com/installing-grunt

http://httpd.apache.org/docs/2.2/install.html

https://docs.oracle.com/javase/7/docs/webnotes/install/

https://github.com/gsi-upm/demo-smartopendata/



<#smartopendata> rdf:type ja:RDFDataset ; rdfs:label "Smart Open Data" ; ja:defaultGraph [ rdfs:label "Smart Open Data" ; ja:MemoryModel ; ja:graphName <http://sod.localhost/sod> ; ja:content [ja:externalContent <file: {$path}/demo-smartopendata/data/sod.rdf>] ; ] ;

Listing 8 Description for the involved Dataset.

3. SEFARAD CONFIGURATION

Make the following changes to the mvvs.js file located in {$path}/demo-smartopendata/js

1. Edit the application-context section by adding a description for the application to be deployed. Listing 20 shows the values to be included.

contextResource= "SmartOpenDataDemo" ; applicationName= "SmartOpenData" ;

Listing 9 Context resource and application name definitions.

2. Edit the query-definition section by adding the SPARQL query and URLENDPOINT to be used. Listing 21 shows the values to be included.

Urlendpoint= "http://localhost:3030/sod/query" ; Sparqlquery= " PREFIX drf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX j.0: <http://inspire.jrc.ec.europa.eu/schemas/gn/3.0/> PREFIX j.1: <http://inspire.jrc.ec.europa.eu/schemas/ps/3.0/> PREFIX j.2: ?res j.3:hasGeometry ?fGeom . ?fGeom j.3:asWKT ?fWKT . ?res j.1:siteProtectionClassification ?spc . ?res j.1:LegalFoundationDate ?lfd . ?res j.1:LegalFoundationDocument ?lfdoc . ?res j.1:inspireId ?inspire . ?inspire j.2:namespace ?namespace . " ;

Listing 10 URL endpoint and sparql query definitions.

4. GRUNT CONFIGURATION

Move the grunt.js file located in {$path}/grunt directory to {$pathgrunt}.

$ pathgrunt corresponds to the grunt directory of your system.

Next, generate a SEFARAD binary package by typing the commands listed below:



%shell%> cd {$pathgrunt} %shell%> grunt smartopendataapp

Listing 11 Command for generating the binary package.

To check if the SmartOpenData app was successfully deployed, access the URL listed below:

http:/$SERVER$/demosmartopendata/index.html#/sparql/slovakiaPolygonsDemo

Listing 12 URL for accessing the deployed SmartOpenData APP.

Where:

$SERVER$ corresponds the server in which you have installed the apache.

/demosmartopendata/ corresponds to apache web directory used for deploy the application.

Now, you should be on the webpage shown on the figure 30.

Figure 10 An overview of SmartOpenData app in SEFARAD tool



4 User manual

This section explains the functionalities provided by the SmartOpenData tool for exploring and visualizing linked data resources. This section describes how users can use the main features of SEFARAD, including all those relative to add widgets, manage the data with different filtering methods, display the final results in some graphical widgets and use the SPARQL editor.

4.1 Add widgets

SEFARAD provides a semantic front-end to Linked Open Data which allows users to visualize, explore and analyse graphically the different features, attributes and relationships of the queried data. Different kind of widgets such as: numeric filter, tag cloud, gauge and maps can be used from SEFARAD. To add a widget please follow the steps described below.

Use the login form located on top-right corner of the browser to be logged into the dashboard system. Use “admin” and “1234” as user and password respectively.

Next, push the button “Add widget”. A window is shown for selecting the widgets. Now, you should be on the webpage shown in figure 31.

Choose a map and a tag widget template. Chosen widgets should appear on the dashboard.

Figure 11 Available widget templates in SEFARAD



4.2 Faceted search

SEFARAD provides an interesting and useful feature for applying multiple filtering options on geo-spatial data. Faceted browsing capabilities can be applied by adding an accordion tag cloud widget and map widget. For enabling the filter option please follow the steps described below.

Add a tag cloud and map widget by following the steps described in section 8.1

Next, select a facet to be applied. Once a facet is selected, it is activated immediately and changes are shown on the map. Figure 32 shows an example about how users can explore geo-spatial data by using faceted browsing features.

Next, if it were necessary, users can apply filtering options on properties previously filtered.

Figure 12 Enabling faceted browsing feature for geo-spatial data.

4.3 SPARQL editor

SEFARAD provides a human-readable SPARQL editor for processing geo linked data queries. It is basically an editor in which users can customize SPARQL queries according to the geo-spatial parameters defined in the data resources. This feature provides a simple syntax highlighted text area bundled with features such as auto completion and the option to choose geo-spatial parameters for customizing queries. Current version involves a set of predefined human-readable queries. However, we are planning to include in the next release a feature in which human-readable queries can be defined and tailored by users. In order to process SPARQL queries by using the SPARQL editor please follow the steps described below.

Use the SPARQL editor located on top-center and click on it. Select a type of query from the combo box.

Next, select the parameters to be included on the selected query. A table result is generated for showing the properties returned. Figure 33 shows an example about an executed query by using the SPARQL editor.



Figure 13 SPARQL editor



5 Developer manual

This section explains how to use and configure widget features on the SmartOpenData application. SEFARAD provides users the capability to create own widgets without being worried about they are going to display the correct information. Thanks to the Knockout.js framework, it is really easy to develop widgets without consider how the filtered data and updating mechanisms can be instanced. To deploy the SmartOpenData application by using a map widget, please follow the steps described below.

1. CONFIGURATING THE SEFARAD TOOL AND FUSEKI SERVER

Configure the fuseki server, the SEFARAD tool and grunt utility by following the procedures explained in steps 2, 3 and 4 of section 7.2.

2. CREATING A MAP WIDGET

Create a file named smartopendatawidget.js in the {$path}/demo-smartopendata/d3 directory and make the following changes:

Create a section named general configuration by adding a description for the type and widget category. Listing 24 shows the values to be included.

// <#general configuration> var widgetName = { // Widget name. name: "SmartOpenData Widget", // Widget description. description: "Map widget for the smart open data", // Path to the image of the widget. img: "img/widgets/widgetMap.png", // Type of the widget. type: "openlayersMap",

}

Listing 13 general configuration section on smartopendatawidget.js file

Add a section for describing how the widget can be renderized. This section should describe how objects can detect and respond to changes. Listing 25 shows the values to be included.

render: function (loc) { if(loc != 'Left' && loc != 'Right') loc = 'Left'; var id = 'A' + Math.floor(Math.random() * 10001); var configid = 'A' + Math.floor(Math.random() * 10001); var field = openlayersMap.field || ""; var properties = { "id": ko.observable(id), "configid": ko.observable(configid), "title": ko.observable(openlayersMap.name),



"type": ko.observable(openlayersMap.type), "field": ko.observable(field), "collapsed": ko.observable(false), "showWidgetHelp": ko.observable(false), "help": ko.observable(openlayersMap.help), "showWidgetConfiguration": ko.observable(true) }; vm.addNewWidget(properties, loc); openlayersMap.paintConfig(configid); openlayersMap.paint(id);

Listing 14 render configuration section on smartopendatawidget.js file

3. CUSTOMIZING THE MAP WIDGET CONTENT

Include the functions for controlling the map widget by adding the

core code inside the paint method. Listing 26 shows a partial view

of the values to be included. Please consider checking the github

smartopendata wiki for more details.

map = new OpenLayers.Map(map_div.node(), { // Resoluties (pixels per meter) van de zoomniveaus: resolutions: [3440.64, 1720.32, 860.16, 430.08, 215.04, 107.52, 53.76, 26.88, 13.44, 6.72, 3.36, 1.68, 0.84, 0.42, 0.21], //units: 'm', controls: [ new OpenLayers.Control.TouchNavigation({ dragPanOptions: { enableKinetic: true } }), new OpenLayers.Control.MousePosition({ prefix: '<a target="_blank" ' + 'href="http://spatialreference.org/ref/epsg/4326/">' + 'EPSG:4326</a> coordinates: ', separator: ' | ', numDigits: 2 }), new OpenLayers.Control.PanZoom(), new OpenLayers.Control.Navigation() ], projection: new OpenLayers.Projection("EPSG:3857"), displayProjection: new OpenLayers.Projection("EPSG:4326") });

Listing 15 paint method on templatewidget.js file

wpx - dn - smartopendata · sites. upon discussion with domain experts, mapping between individual...

Documents