2 publishable summary€¦ · d6.3 – final project report – periodic report year 2 10 / 85 2...

D6.3 – Final Project Report – Periodic Report Year 2

10/85

2 PublishableSummary

2.1 ContextandObjectivesIt is broadly acknowledged that a unified solution for transforming and renovating existing datasources, regardless of the original data format, would greatly enhance the ability of publicorganisationstoprovideusable,machine-processablelinkeddata,whileofferingSMEstheopportunityto combine and link existing public sector information with privately-owned data in the mostresourcefulandcost-effectivemanner.Towardsthisdirection,however,thereisalsoastrongneedforsupporting consumers unfamiliar with the linked data paradigm through interfaces that hide theunderlying complexity and allow the re-use of existing software apps and database managementsystems.LinDAaimstoassistSMEsanddataprovidersinrenovatingpublicsectorinformation,analysingandinterlinking with enterprise data by developing an integrated, cross-platform, extensible softwareinfrastructure, titled as the “LinDAWorkbench” that handles end-to-end the transition of a data-powered enterprise to a linked data-powered organisation. The LinDA Workbench allows for thetransformationofvariousformatsofdataintoarbitraryRDFgraphs,theconstructionofLinkedDataqueriesthroughuserfriendlyinterfacesaddressedtoSMEs,includingintuitivevisualisationmethodsandcharts,whileitisinapositiontoperformfurtherstatisticalanalysistothequeriesresultsbasedon the R framework. The LinDA Workbench can be used either as a Service, or be deployed andoperatedasa standalonesolution,whileusersare inaposition tomakeuseof its sub-modules inadistinctmanneralso.The followingbulletsdescribe themaincomponentsof theLinDAWorkbench,whichare:

• A data transformation solution that provides a simplified workflow for renovating andconverting a set of common data containers, structures and formats into RDF graphs. Thetransformation engine can be used to develop custom solutions for SMEs and public sectororganisations or be integrated into existing open data applications, in order to support thesemi-automatedconversionofdataintolinkeddata.TheoverallplatformallowstheexportofarbitraryRDFgraphsastabulardata,supportingSMEstostorethefinalresultsofdatalinkingintorelationaldatabasesorprocessfurtherwithspreadsheetanddataanalysissoftware.

• ArepositoryforaccessingandsharingLinked-DatavocabulariesandmetadataamongstSMEsthatcanbelinkedtotheLOD(LinkedOpenData)cloud.ThesystemallowsSMEstoreferenceand enrich metadata shared by well-established vocabulary catalogues (LOV, prefix.cc,LODStats), thus contributing to easy and efficientmappingof existingdata structures to theRDFformataswellastoincreasingthesemanticinteroperabilityoftheSMEsdatasets.

• The Query Designer and the Query Builder tools which enable non-experts to formulate aSPARQL query and explore open datasets in an innovative and easy way,to use graphicalmethodstointeractivelybuildasimpleorcomplexqueryovermultipledatasourcesandviewtheresults inaSPARQLeditor.TheQueryDesigner follows theparadigmandqualityofSQLQuery designers of popular relational database management systems where with simpledrag’n’dropfunctionalityuserscanperformcomplexSPARQLquerieswhiletheQueryBuilderofferssimilarfunctionalitythroughawizard-likeguidedlistprocedure.

• The LinDAVisualization engines that can help enterprise users gain insight from the linkeddata that the company generates. With this engine users can visualize data in linked dataformat taking into advantage their semantics. The LinDA visualization provides a largelyautomaticvisualizationworkflowthatenablesSMEstovisualizedataindifferentformatsandmodalities. In order to achieve this, a generic web application is being developed based onstate-of-the-artLinkedDataapproachestoallowforvisualizingdifferentcategoriesofdata,e.g.statistical, geographical, temporal, arbitrary data, and a largely automatic visualizationworkflowformatchingandbindingdatatovisualizations.

• TheAnalytics andDataMiningComponent supports the realizationof analysisbasedon theconsumption and production of linked data. A library of basic and robust data analyticsfunctionalityisprovidedthroughthesupportofasetofalgorithms,enablingorganizationsand


11/85

enterprises to utilize and share analytic methods on linked data for the discovery andcommunicationofmeaningfulnewpatternsthatwereunattainableorhiddeninthepreviousisolateddatastructures.Theanalyticsanddataminingcomponent isbasedonanextensibleandmodulararchitecturethatfacilitatestheintegrationofalgorithmsonaperrequestbasis.The development of the component is based on open-source software while integration ofalgorithmsisbasedonopen-sourceanalyticsprojects(mainly,theRstatisticsproject).

• A set of Linked Data Consumption Applications (ConsApps), upholding the linked dataprocesses andprovidingend-to-end solutions for theendusers’uniquebusinessworkflows.ConsApps regard a set of applications that are developed aiming to provide to end users(includingpilotusers)functionalitiesthatarenotprovidedthroughtheLinDAWorkbench.Theobjectiveistofacilitate,throughspecificapplications,thedailybusinessprocessesoftheSMEsbased on the redesignedworkflows that take into account the usage of the LinDA tools. Assuch,ConsAppscouldbeconsideredassmalltailormadesolutions,easy-to-implementandoflow-cost,servingthespecificneedsoftheLinDAend-usersthatoccurswhileinteractingwithLinDAWorkbench. In thiswayLinDAWorkbenchcould justify itspromise tobea complete,end-to-endsolutionfortheincorporationofthelinkeddataconceptswithintheSMEs.

Figure2-1:TheLinDAConcept

ThethreepilotsdevelopedduringtheLinDAprojectarethefollowing:• Linked Data Analytics in Business Intelligence (CP Pilot) - This pilot tackled the domain of

businessintelligenceanalyticsandexaminedtheeffectsofvarioussocioeconomicparametersintheOver-The-Counter(OTC)medicinesmarketinEU-28countries.

• LinkedDataAnalytics in theEnvironmental Sector (HYPERBOREAPilot) –Thispilot tackledthe domain of environmental analytics and examined the health impact of outdoor airpollutionininternational,nationalandregionallevel.

• LinkedDataAnalytics in theMedia Industry(PIKSandTTNEWS24Pilot)–Thispilot tackledthe domain of investigative journalism and media analytics and provided an approach forknowledge extraction over post production scripts, as well as the production of mediaanalytics,targetingmainlyatfacilitatingmarketingactivities.

Moreover, during the course of the project, the consortium partners used their client portfolio,alongside as dissemination andmarketing activities to organise events andworkshopswhere SMEs


12/85

were introduced to the LinDA project, learned about the benefits of Linked Data, tested the LinDAWorkbenchanddeclaredtheirinteresttoadopttheLinDAsolutioninduetime.TheoverallrealisationoftheLinDAprojecthasbeenachievedthroughtherealisationofthefollowingobjectives:

• Objective1:Enhancetheabilityofdataproviders,especiallypublicorganisationstoprovidere-usable,machine-processablelinkeddata.

• Objective 2: Provide out-of-the-box software components and analytic tools for SMEs thatoffer the opportunity to combine and link existing public sector informationwith privately-owneddatainthemostresourcefulandcost-effectivemanner.

• Objective3:DeliveranecosystemofLinkedDataPublicationandConsumptionApplicationsthatcanbeboundtogetherindynamicandunforeseenways.

• Objective4:Demonstrate the feasibility and impactof theLinDAapproach in theEuropeanSMEsSector,overasetofpilotapplications.

• Objective 5: Achieve international recognition and spread excellence for the researchundertaken during the LinDA implementation towards enterprises, scientific communities,dataprovidersandend-users.Diffuseandcommunicatereadily-exploitableprojectresults,ofapro-normativenature.Contributetostandardisationandeducation.


13/85

2.2 ConsortiumTheconsortiumofLinDAconsistsof7partnerscomingfrom4EuropeanCountries.Thepartnersoftheconsortiumareshownbelow.

NATIONALTECHNICALUNIVERSITYOFATHENSDECISIONSUPPORTSYSTEMSLABORATORY

(NTUA-DSSLab)

Co-ordinator

Greece

FRAUNHOFER-GESELLSCHAFTZURFOERDERUNGDERANGEWANDTENFORSCHUNGE.V.(FOKUS) Germany

GIOUMPITEKMELETISCHEDIASMOSYLOPOIISIKAI

POLISIERGONPLIROFORIKISETAIRIAPERIORISMENISEFTHYNIS(UBITECH)

Greece

UNIVERSITYOFBONN(UBONN) Germany

PIKSELLIMITED(PIKS) UnitedKingdom

CRITICALPUBLICSLTD(CP) UnitedKingdom

HYPERBOREAS.R.L.(HYPERBOREA) Italy

TTNEWS24S.R.L.(TTNEWS24) Italy


14/85

2.3 WorkPerformedandResultsAchievedDuring the second year of the project the main focus of the LinDA project has shifted from coredevelopment to widespread dissemination, by organising a series of events to showcase itsinnovations and to provide hands-on demonstrations to interested SMEs and stakeholders,while italsoparticipated in a series of international events, of both academic andbusinessnature trying todiffuseknowledgeandpromotetheLinDAWorkbench.Moreover, during the Y2 of the project, the 2nd version of the LinDA tools has been developed anddeployedinthecommonenvironmentthatexistedfromY1,theLinDAWorkbench.Theenhancementof theLinDAtools,wasperformedbasedontherefineduserrequirementsandusagescenarios thatderivedmainlythroughtheevaluationandtestingperformedinthepilotsduringY2aswellasonad-hoccommentsandfeedbackacquiredduringtheinteractionofthetechnicalteamwiththepilotusers.Furthermore,thetoolshavebeenalsoupdatedtoreflectthechangesintheLODlandscape,tomeetthedemandsofmoreSMEsandstakeholdersingeneral,asdocumentedduringtheLinDAworkshopsandevents,whilefurtherfeaturesthatwereinthebacklogoftheprojecthavebeenimplemented.During the2nd year, following theupdatesof the tools, thepilot’s operationplanhasbeenupdated,includingnewscenariosandthematicareas,andtheevaluationframeworkwasputintooperationtorecordtheperformanceoftheLinDApilotsinordertoexploreitsaddedvalueandpointsthatneedtobe reconsidered, towards fine-tuning and further optimising the tools.Last but not least, theconsortium defined the exploitation plan of the solution, which is focused mostly around internalexploitationactivities tobeperformedby thepartners,whileasustainabilityplanhasbeenbroughtforward, offering the overall LinDA solution as open source and the LinDA consortium committingthrough a MoU, so as to sustain the solution in the forthcoming years. In this respect, the LiNDAsolutionandtheoverall toolswillbemaintainedbyallpartners(eachonebeingresponsible for thepartthathehasimplemented)foraperiodofatleast2years,withpartnerscommittingtoworkinabest-effortmannertoreleaseupdatesandbugfixesasnecessary.TheLinDAtoolshavebeenplacedinGithub in order to be available to all interested parties,while a help-desk via email is operating toestablishasupportchannelwithinterestedstakeholderswhowouldliketotrythetools.Also,LinDAhas came to an agreementwith other projects such as ODINE and YourDataStories for offering theLinDAtoolsasanoptiontobeusedfromthoseprojects’stakeholders.Ingeneral,theactivitiesoftheLinDAapproachhaveproceededaccordingtoplanandasperDoWandhaveproducedsignificantresultswhicharesummarizedinthecomingcategory:UpdatedUserRequirementsandBusinessScenariosDuringthe2ndyearoftheLinDAprojecttheinitialuserrequirementsandbusinessscenariosthatsetthe groundwork for the creation of the LinDA tools in Y2 have undergone an update to reflect thenecessarychangesbothatthetoolsbutalsoatthepilots’level.Morespecificallythekeyachievementsforthe2ndyearare:

• Definition of final 7 business scenarios for the utilization of Linked Data in the domains ofBusinessIntelligence,EnvironmentandMedia.Thescenarioschosenarethoseof:Reputationmanagement and Communication strategy, Press monitoring reports, Publishingenvironmental data, Health impact of pollution in different areas, Sharing journalistic data,Data mining of Post Production Scripts (PPS), Video content and delivery managementanalytics.

• Theupdatedbusinessscenariosgeneratedan initialsetof20userstories inviewof formingthebaseelementsfortheLinDAproject.

• Anupdatedandprioritisedlistoftechnicalfunctionalandnon-functionalrequirementsdrivenbytheupdatedbusinessscenariosanduserstories.TherequirementshavebeenrepresentedandmanagedasGithubissuesfortheefficientmanagementoftheLinDAtoolsdevelopment.

• AcompletesetofAcceptanceCriteriaandAcceptanceTestingProcedurehavebeendefined.• Anupdateofthestateoftheartanalysisonexistingopensource/commercialmethodsand

componentsthatcanarecomparedtoorcouldbe integratedintotheLinDAWorkbench.Foreach of the Linked Data components a respective test-bed environment has been setup in


15/85

ordertoanalyseandreporttheirbenefits,capabilities,shortcomingsandlimitations(e.g.easeofintegration,licensingissues,complexity).

• AnupdateoftheonlineLandscapeofLinkedDataTools4sectionintheLinDAprojectwebsitethatprovidesa convenientoverviewof the state-of-the-art analysis, aswell as amuchmoreefficientmaintenanceandupdateoftheanalysedlinkeddatatools.

LinDAWorkbenchDevelopmentDuring the 2nd year of the LinDAWorkbench implementation, the following key achievements havebeenreached:

• The LinDA Workbench, the integrated environment for hosting the LinDA tools has beenupdated with new features, including the ability to add off-the-shelf popular SPARQLendpointsonthedatasourceslistaswellastomonitortheperformanceandresponsetimesofremote data sources. The LinDAWorkbench facilitates theworkflow between the tools andhandlesthemaincommunicationwithaselectedtriplestore.

• TheLinDATransformationEnginehasbeensubstantiallyrevisedandintegratedtotheLinDAWorkbench.Theenginecanbeusedtosupportthemappingandtransformationoftraditionaldata structures and formats into linked data. To this end, the Transformation Engine hasfocusedontwowidelyuseddatasourcesa)relationaldatafromdatabasessuchasMySQL,andb)tabulardatafrom.CSVfilesandExcelspreadsheets.Thenewversionoftheenginesupportsan easier transformation workflow, more flexible vocabulary matching as well as a NamedEntityRecognitionserviceinordertoproduce5-starslinkeddata.

• The final versionof theLinDAVocabularyandMetadataRepositoryhasbeendelivered.TheLinDA Vocabulary andMetadata Repository leverages and syncswith existing online linkeddata vocabulary services (LOV,prefix.cc, LODstats) inorder to assist users andSMEsduringthe data transformation process to select appropriate vocabularies (in terms of popularity,ratingandrelevancetothespecificdomain/industry)forthesemanticrepresentationoftheircurrentdatastructurestotheRDFformat.A“SuggestAPI”hasbeendevelopedthatperformsautomaticand intelligentmappingsuggestionsbasedontheLinDAVocabularytobeusedbyexternalapps,includingtheLinDATransformationEngine.

• The final versionof theRDF2AnyAPI fordata transformation fromLinkedData format to anumberofformatsincludingRDB,XML,CSVandJSON.

• The final version of theQueryBuilder tool that enables non-experts to formulate a SPARQLqueryandexploreopendatasets.

• The final version of theQueryDesigner tool has beendelivered that provides an innovativeandeasywaytousegraphicalmethodstointeractivelybuildasimpleorcomplexqueryovermultipledatasourcesandviewtheresultsinaSPARQLeditor.TheQueryDesignerfollowstheparadigm and quality of SQL Query designers of popular relational database managementsystems(Oracle,SQLServer,etc.)butseamlesslyadjusted toharness thepotentialofLinkeddata.With simple drag’n’drop functionality users canperform complex SPARQLqueries andfilteringincludinginterlinkingwithexternalSPARQLendpointsthroughtheuseofSPARQL1.1Federated Query. The final version includes user interface enhancements , scalability andperformance improvements as well as many more features including support for SPARQLaggregations.

• ThefinalversionoftheLinDASPARQLeditorthatprovidesfunctionalityofatext-basedquerywizard over linked data. More specifically the LinDA SPARQL editor provides code styleformatting and intelligent code completion (by clicking CTRL-space) for suggesting SPARQLsyntax,namespaces,availableendpoints,classesandproperties.

• The final version of the LinDA Visualization and Exploration system, that allows differentvisual representations of data sources in the Linked Data format and provides automaticrecommendations for determining the compatibility between the selected dataset and theavailablevisualisationsandsuggestingalistofvisualisationsaccordingly.

4 Landscape of Linked Data Tools - http://linda-project.eu/linked-data-tools


16/85

• The final version of the LinDA Analytics Engine has been made available, based on the Rlanguage,thatallowfortheconstructionandprocessingofanalyticalgorithmsandproceduresregardingdatacomingoutofLinkedDatasets,alongwiththedefinitionandusageofaLinkedDataAnalyticsOntologyandarelevantinterlinkingpolicy.

LinDAPilotsOperationandEvaluationY2involvedtheoperationoftheLinDApilotsatfullscale,testingtheLinDAWorkbenchintheirdailyoperationsandestablishingadirectcommunicationchannelwiththetechnicalteamtofine-tunethetools, optimise the datasets, integrate the ConsApps and report back on the impact from using thesolution.Morespecificallythefollowingtaskshavebeenperformed:

• Three pilots have been operational with the direct contribution of the pilot users - SMEs,namely the Business Intelligence Analytics (BIA) pilot execured by CP, the EnvironmentalAnalyticspilot,executedbyHYPERBOREAandtheMediaAnalyticspilot,executedinitiallybyTTNEWS24andlaterbyPIKS,withinitsenvironment.

• An updated description of the LinDA pilots and details regarding their operation and theirevaluationhavebeenexaminedanddocumented.

• Thein-depthanalysisofY1hasbeencontinuedinordertoa)redesignthebusinessprocessesthatwererequiredbasedontheuseoftheLinDAtools,b)identifythefinalsetofdatasourcesthathavetobecollectedandalsocreatedandinterlinkedfortheoperationofthepilots,basedontheupdatedscenariosdevisedinY2,c)defineandrealisethetypeoftheanalysisalongwiththe selected algorithms and visualisations that correspond to the new requirements andscenarios.

• AsetofeightConsumptionApplications(ConsApps)havebeendeveloped,whichcanbeseenasthetailor-made,additionalsolutions-applicationsthatcanaccompanythecoreLinDAtools,inorder toaddress the specific,uniqueneedsofdifferentpilots,whilepromoting theLinDAWorkbenchtoamodernOne-Stop-ShopforLinkedDataManagement.

• Theconstantevaluationofthetechnicalsolution(LinDA’sWorkbenchelementsandplatformasawhole)andof theoperationof thepilots toshowtheaddedvalueachievedthroughtheproductionandconsumptionoflinkeddatafortherealisationofanalysisbasedontheneedsoftheinvolvedSMEsintheproject.

Networking,Dissemination,ExploitationandSustainabilityDuring the 2nd year, LinDA dissemination, engagement and exploitation tasks was intensified andfocusedtothefollowingtasks:

• A revamped version of the LinDA web site has been launched for the dissemination of theproject’sresults, focusingmoreonthe finalproductand lessontheproject itself, inordertosupporttheexploitationandsustainabilityplan.

• Constantpublicationson5socialchannelstomaintaintheusers’interestaliveanddrivetraffictothewebsite(Facebook,Twitter,Google+,Youtube,Slideshare).

• Release of commercial-oriented videos and teaser videos explaining the LinDA innovationsthathasbeenusedtodrawattentionofaudience.

• Liaisonandcollaborationactivitieswith14relatedprojects.• Establishmentofcollaborationagreements(signedMoUs)with4projectsintheOpenDataand

LOD field (ODINE, FREME, OpenBudgets, YourDataStories), which greatly affect thesustainabilityofLinDAas theproject’soutputsarealreadybeingrecommendedandusedbysomeofthoseprojects.

• Further Online and printed material (leaflets, brochures poster, tutorials, etc.) has beencreatedfordisseminationpurposes.

• Activeparticipationto12conferencesandworkshopswithLinDApresentationsandposter.• 9academicpapersintotal(2inacademicjournalsand7inconferences)• Organizationof2projectspecificWorkshopstargetingSMEs:


17/85

o “VerlinkteDaten -MehrAusDenEigenenUnternehmensdatenHerausholen”on17thSeptember 2015 hosted in Berlin, in conjunction with eBUSINESSLOTSE 2015. Thisevent was a Technical Workshop organised by FOKUS, at the Fraunhofer FOKUSpremises in order to publicly disseminate the results of the development, engagepotentialusersandinterestedpartiesandcollectfeedbackaboutthetools.

o “LinDA:LinkedDataandAnalyticsforSMEs”on20thNovember2015hostedinAthens,organisedbyNTUA,UBITECHandCP.ThegoaloftheworkshopwastoshowcasetheLinDA tools, test the usability and performance of the tools, and involve interestedSMEsparticipantsintheusageanddevelopmentoftheopensourcetools.

• Organisationof1HackathonEvent,inconjunctionwiththe“LinDA:LinkedDataandAnalyticsforSMEs”Workshop,heldinInnovathenstargetingsoftwaredevelopers.

• Updates on the project exploitation and sustainability plans to finalise the set of exploitableassetsandtodefine theprojectexploitationandsustainabilitypaths tobe followedafter theendoftheproject.

Allpublicdeliverablesoftheproductcanbeaccessedonlineat:http://linda-project.eu/deliverables/

Table2-1:KeyPublicLinDAdeliverablesinYear2

LinDAKeyPublicDeliverablesforthe2ndYearD1.3-LinkedDataComponentsandToolsSotAandBusinessScenariosforLinkedDataUtilization-FinalVersionD2.3-LinDATransformationEngine,LinkedDataVocabulariesandMetadataRepository-FinalVersionD3.2-LinDAPublicationandConsumptionFrameworks,LibrariesandInterfaces-FinalReleaseD4.4–Pilots'LinkeddatasetsandConsumptionApps-FinalreleaseD4.5-PilotsResultsandEvaluationReport-FinalVersionD5.6-DisseminationActivitiesReport-FinalVersionD5.8-Web-enabled(orDVDbased)PublicShowcase


18/85

2.4 WebsiteandSocialMediaChannelsDuring the second year of the project, the website that resides at http://www.linda-project.eu hasbeenconstantlyupdatedwithnewsaroundtheproject,inordertoshowcasethedevelopmentsoftheconsortiumandalsodisseminateevents likewebinars,workshopsandparticipation in internationalconferences/eventswhereinterestedstakeholderswouldbeinapositiontoreachtheLinDAprojectandobtaininformationaboutitinadirectmanner.Thesitecomplementedbytheactiveoperationofsocialchannelstomaintaintheusers’interestaliveanddrivetraffictothewebsite.Themostimportantonesusedarethefollowing:

• Facebook(news),https://www.facebook.com/LinDAFP7• Twitter(news),https://twitter.com/LinDa_FP7• YouTube(videos),https://www.youtube.com/channel/UCfZHhkxIN_O1jRovE2lTWwA• SlideShare(presentations),http://www.slideshare.net/LinDa_FP7

Close to the end of the project, the LinDA website has been redesigned in order to support thesustainabilityoftheproject’sresultsandpotentialexploitationopportunities,havingamoreproduct-centric presentation than a research project presentation. The following screenshot present thelandingofthewebsiteatitspresentstate.

Figure2-2:LinDA’sUpdatedWebsiteLandingPage

ThepublicwebsiteoftheprojectisoperatedbyNTUA,andwillbemaintainedatoperationalstatusbyNTUAforatleast2yearsaftertheendoftheproject.

2 publishable summary€¦ · d6.3 – final project report – periodic report year 2 10 / 85 2...

Documents