10Article

By Aleksey Borodulin and

Aleksandr Kleshnin

Open Source GIS Software

The idea of using open software atIntetics was first aired in May 2011.At that time the main working toolsfor Intetics’ specialists was propri-etary software provided by the client.

The volume of manual vectorization projectswas increasing along with the growing num-ber of employees. We planned to hire approx-imately 50 new employees; though the timespent using GIS software did not exceed 5-16% for a single employee. As an alternative

to purchasing extra proprietary licenses, adecision to use open software was made.

Briefly about the projectThe client provided us with shp-files contain-ing road graph and building geometry. In thecourse of the project, we had to clarify theclient’s road graph and buildings contoursspatial positions data. The first step was toclarify existing geometry, and then add thenew one. After completing this, we followed

the same procedure with the building poly-gons.

Choosing and implementing softwareAfter careful evaluation we chose gvSIG andQuantum GIS, mainly because these productsare widely available as desktop GIS software.Then we completed several test projects usingboth gvSIG and QGIS and compared theresults and the expert reviews. Generally,gvSIG performed better, although some tools

U s e r E x p e r i e n c e s a t I n t e t i c s

Since the mid 2000s open GIS has gradually been attracting the attention ofbusinesses and government agencies around the world. Nevertheless, the introduction of open GIS in business processes is hampered by the fact that atthe moment there is no ready-made method of implementation for open GISsolutions. In this article we want to share the experience of our implementationand the use of open GIS software at Intetics.

March 2013

Intetics employees

Prod_GEO213_Prod GEO66 14-02-13 19:30 Pagina 10

11

Latest News? Visit www.geoinformatics.com

were less user-friendly. Among QGIS strengthsare nice and logical GUI, multiframe support,high pace of development, an active users’and developers’ community, lots of plug-insand rather good documentation. Weaknessesinclude slow data panning, slow raster render-ing and overall low performance and instabili-ty.Though QGIS performance was not so good,it was finally chosen as GUI as ease of mas-tering played a very important role for us. Dueto the specific features of the project, softwareperformance did not influence the overall workflow much. At the same time, software simplic-ity and GUI consistency is very important dur-ing the first exposure to open GIS software, asGUI ease minimizes users’ discomfort. The difficulties with Quantum GIS began withthe lack of network software installation tools.Initially users had to install and configure thesoftware all by themselves. It was time con-suming and initiated a torrent of questions fromusers. Beginning with the 1.7.3 release westarted using the portable QGIS version as itsconfiguration was set and all the necessaryplug-ins loaded by default. The portable ver-sion is distributed by copying.The 1.7.0, 1.7.3, 1.7.4, 1.8.0 QGIS releaseswere used in the project. Whilst preparing forthe project, the new releases were tested anda changes analysis made. The decision to useit or not was made based on the data collect-ed; evaluation of all the advantages and dis-advantages of new releases compared to thecurrent version. Assuming a positive decision,we performed pilot works using the newrelease of the software.Issues that appeared during QGIS utilizationand new version releases:• shp-files merge didn’t work in 1.7.4 (fixed

in 1.8.0);• snapping to segment didn’t work in 1.7.4

(fixed in 1.8.0);• Cyrillic symbols weren’t displayed in attribute

table in 1.8.0. We imported layers whichcontained Cyrillic attributes in DB usingQGIS version 1.7.4.

According to the requirements specification,the road graph should not have geometrygaps and overlaps. During the project our spe-cialists mainly worked on separate distancedterritories and the amount of geometry situat-ed on the borders of the neighboring workingareas was minimal. Vectoring data was storedas shp-files on the network drive. When theediting was completed, data from separateshp-files were combined into one, commongeometry on borders was checked and man-ual editing was performed if needed.As the project developed, we began vector-

ization of large urbanized territories andfaced the problem of users’ interaction inworking areas which had common geometry.The solution was to create a common infor-mation space using DB.Initially, we considered DB implementationusing SQLite/SpatiaLite or Postgre SQL/PostGIS. Some tests were done and we dis-covered SpatiaLite performance is significant-ly reduced when used by many users simulta-neously; this was the reason for declining it.PostGIS performance met our requirementscompletely. During the preparation process, databaseengineer imports pre-processed source datato the DB and editing rights were adjustedusing pgAdmin application and DatabaseManager Plug-in. PostGIS allowed us to coor-dinate the work on borderline areas and per-form quality control rapidly. It allowed us toevaluate current progress in particular work-ing areas and the overall project progress(SQL queries on the number of polygonalobjects and  the sum of road graph edgeslengths), derive performance index, performstatistics calculation and stream monitoring

(project manager is able to see any work areaany time). The use of SQL queries allowed usto perform DB layers merge and copying andcreate intersection layer between two layers.Creating DB layer copies using SQL queriesand DB layer data unload to local drive usingOGR library is also possible.In the first stages of DB usage we performedvectorization for several large and geo-graphically distanced working areas. Tworoad layers were created in the DB – for thenew and amended geometry. Problems inthe QGIS work appeared when a large num-ber of objects were created in one layer(more than 60 thousand for a linear layer),for example, long data loading, delays inpanning, layer attributes table opening,object selection, operation performanceusing field calculator, etc. We decided,therefore, to create several layer pairs forevery working area instead of using two layers for all the areas. This meant that thenumber of records for every table  in the DB remained supportable even at the endof the process and did not cause loss of efficiency.

Implementation process

QGIS Plugins

Prod_GEO213_Prod GEO66 14-02-13 19:30 Pagina 11

12

Our peopleFor Quantum GIS there is a fairly complete man-ual in Russian (translation done by gis-lab.infocommunity). However, for staff training onQGIS, an internal guide was prepared. Thisdescribed basic functionality and the processesof typical project tasks. A users’ and develop-er’s community support and official documenta-tion were used for the internal guide. We wantto express thanks to all who supported us andspecial thanks to the GIS-Lab community. Writteninstructions were tested by experts and wereamended in the process as a result of experi-ence gained.Due to the peculiarities of the technological pro-cess, vectorization (activity performed with thehelp of QGIS) is done at Intetics every quarter.The time in between was used for analysis andprocess changes introduction.Implementation and maintenance of the newprocess was lead by the leading experts (LE).LE duties included: analysis of the project expe-rience, working instructions development andimprovement, helping specialists in masteringsoftware and the whole process and solving anyarising problems.All the working documentation was uploadedto the team collaboration software, Confluence.LE edit and add documentation along with workinstructions to the Confluence. Employees useConfluence to ask for document clarification, toadd information (improvement suggestions, tipson dealing with issues) and share experience.Changes are reviewed and moderated by LE.Confluence enables the sharing experience on-line and minimizes the number of meetings.At first we faced employees’ resistance to theinnovation, which manifested itself as inaction.Some employees were not motivated enough tomaster new, sometimes unstable, software.

Often employees compared QGIS and com-mercial GIS functionality (in favour of the latter).The process actively developed, new tools andinstructions were introduced, but a number ofemployees continued working “as before.”These difficulties were not caused by specifictasks or software, however; all engineering com-panies face the same problems in the processof new working tools implementation. In orderto solve such problems, each quarter and priorto the project start, a list of innovations is creat-ed and e-mailed to the employees. Additionally,meetings are held allowing leading experts toexplain the peculiarities of the project. A sepa-rate section in the project info DB is created toshare experience with open source software,both positive and negative (such tips often pre-vent common mistakes being made).Our internal QGIS training is developed foremployees who have at least some experiencewith GIS software and with real-life projects.Normally, with some experience with GIS soft-ware, employees master QGIS to an adequatelevel in 1-2 hours. This was achieved by assign-ing a mentor to every junior developer.Subsequently, the assimilation process for newemployees was optimized.

Technical and organizational specifics ofthe project The preparation process included initial WMS-data quality check. If, for some reason WMScould not be used (slow data loading, bad qual-ity of the image), then pre-loaded satelliteimages in the form of tiles were used. Virtualrasters are created out of tiles, using moduleGDAL.To reduce time spent by the user on provision-ing, qgs-project templates were used. Templateswere created at the preparation stage. The tem-

plate contains all the necessary layers from thePostgreSQL/PostGIS DB (shp-files were suppliedtogether with templates before the DB was cre-ated), WMS layer added, styles/group views,coordinate systems, snapping options andoptions for displaying and editing attribute dataconfiguration set. Templates are distributed bycopying via the network.In the first stages of the project the topologyerrors correction was performed with the QGISGRASS module (snapping of the line vertex inthe nodes inside the specified tolerance, linesbreaking upon intersections, removing pseudo-nodes). We also tried the topology check usinggvSIG, but found it both inconvenient and timeconsuming. At the end of the day we decidedto do all the required checks with GRASS, as itgave us output acceptable to our customer. Thefollowing was done: search for lines notsnapped (dangled nodes), removing pseudo-nodes, lines self-intersection correction andremoving self-overlapped geometry.JIRA is a tracking system our company uses fortask tracking and time management. Territorypolygons marked by the client are divided intosmall-sized working areas. For each area arecord in JIRA is created by PM. Every recordcontains the following information: the name ofthe region, estimated time required for vector-ization, the deadline and name of the employ-ee assigned. JIRA allows ongoing monitoringdaily (status, readiness percentage and the timespent) and statistic gathering.

ConclusionWe successfully implemented the execution oflarge-scale works using open source softwarecombined with the proprietary software. Wenow have a well-established documented pro-duction process. We plan to develop this process by implement-ing the following steps:• Develop an efficient bug reporting process;• Implement topology processing using PostGIS;• Create local service of raster data on the

Geoserver basis;• Finalize the transition process to newer soft-

ware releases.Intetic’s example shows the real possibility ofsuccessful open source GIS software use forlarge scale projects. Thanks to very detailed pro-cess description and precisely defined teamroles, the project was completed on time andwith the quality level required by customer.

Aleksey Borodulin and Aleksandr Kleshnin are Intetics GIS Analystswith specialization in open source and multiuser database solutions.For more information please visit www.intetics.com/geo or send an

e-mail to geo@intetics.com.

Article

March 2013

Tool suite

Prod_GEO213_Prod GEO66 14-02-13 19:30 Pagina 12