Mobile cultural heritage guide location-aware semantic search

Chris van Aart(1), Bob Wielinga(1,2), Willem Robert van Hage(1)

(1) VU University Amsterdam, the Netherlands (2) Universiteit van Amsterdam, the Netherlands

chris@vanaart.comOctober 2010

Data: http://bit.ly/ekaw2010

Simulator: http://bit.ly/i-ams

iOS: http://bit.ly/lodapp

Recovering of tourism market in

Amsterdamnot fast enough

(Digital) Cultural heritage• Museums• (old)buildings (monuments)• Statues• Art-works• Historic events

<title>De Prinsengracht bij de Reesluis, 1663 - 1664</title>

<keywords>schilderij, stadsgezicht, stadsgezicht Amsterdam, Prinsengracht, Reesluis, scheepvaart</keywords>

<title>Roemer EERSTE STEENLEGGING NIEUWE KERK, 1647</title>

<keywords>drinkglas, roemer, kelkglas, heiplan toren Nieuwe Kerk, steenlegging, Nieuwe Kerk, Willem Backer, Cornelis Backer, Stichting Projecten De Nieuwe Kerk AmsterdamGlascollectie</keywords>

<description>Op de kelk is in diamantgravering het heiplan voor de nooit voltooide toren van de Nieuwe Kerk aangebracht.<description>

<title>Anatomische les van Prof. Frederik Ruysch, 1670</title>

<keywords>schilderij, portret, groepsportret, anatomische les, ambachten en beroepen, onderwijs, Chirurgijnsgilde, Leendert Fruyt, Aert van Swieten, Frederik Ruysch, Gillis Hondekoeter, Rogier de Coen, Joris van Loon Tilanus, Jacob Brandt</keywords>

<description>Frederik Ruysch (1638-1731) was de opvolger van Jan Deijman als praelector van het chirurgijnsgilde.De jonge doctor demonstreert de ligging van de lymfevaten in de lies. De nadruk ligt nu niet op de levenloze staat van de dode, zoals bij Rembrandt. Het lichaam op de snijtafel lijkt eerder toe te behoren aan iemand die slaapt. Waarschijnlijk wilde Ruysch hiermee zijn specialiteit verbeelden: dankzij zijn preparatietechnieken leek het alsof de doden nog leefden. Het schilderij is na restauratie voor het eerst sinds lange tijd weer te zien.</description>

<title>De brand in het oude stadhuis (1652)</title>

<keywords>prent, stadsgezicht, brand van het oude stadhuis op de Dam, Dam</keywords>

http://bit.ly/eculture

Cultural heritage data cloud

http://e-culture.multimedian.nl/resources/datacloud/

http://dbpedia.org/resource/Amsterdam

27

Location awareness / LBS• Real time locating and positioning• Sun/Stars, Compass, Postal code, WGS84 (GPS)… • communication, networks, traffic, logistics, security,

search & rescue, business administration, leisure,…

DBPedia mobile: Google maps

“Louvre app”

Research question “How to find cultural heritage POIs at a

particular location, given the constraints of a mobile device, using current Semantic Web data and technology?”

Challenges1. enrich location data by constructing an

``enriched local map'' of nearby Points of Interest2. Enrich POIs with additional information, such as

e.g. events, persons and artworks.3. present the results in a manageable way on a

mobile device

Approach

• Combine: – general knowledge about geolocations and points of

interest:• GeoNames, LinkedGeodata, Freebase and DBPedia

– specialized knowledge about the cultural heritage domain• Art and Architecture Thesaurus, • Union List of Artist Names, • Netherlands Institute for Art History• Rijksmuseum Amsterdam.

The “Active” tourist: What do I see? Next stop? Hot spots? History? Stories? Events? Persons? Descriptions? Maps?

From Geolocation to Real-world Annotation

Task structure

11

Harvest Locations Iphone GPS

Latitude = 52.368611 Longitude = 4.88944 (we ignore heading)

==> select those geo data that are relevant to this point Searching dbpedia, linkedgeodata, geonames for locations and

objects within a certain distance. Crawling via various relations (owl:sameAs, nearby etc) we gather

a collection of RDF resources related to our location. Using a Prolog spatial indexing package (space) we can compute

distances and directions from our current location.

22

Merging geo info Non trivial:

Different schemas Different labeling conventions Different (GPS) data

Spui in Amsterdam has at least 5 coordinates in LOD There is also an Spui in The Hague

Errors in human annotations Resolve conflicts

Begijnhof isa way Begijnhof isa area

Begijnhof isa building

33

44

rdf:type property (a.o. restaurant, shop, building, church, place of worship, university, way, bequinage, market, marketplace)

linked geo data ontology (LGDV):Facets={ location, event, artwork or people}SubFacets{ painting, photograph, book, artist, musician, politician, sport, conflict,..}

In practice

Spui square (52° 22 7″ N, 4° 53 22″ E′ ′ )

What does LOD return?

• Linked Geo Data– Spui25, Nieuwezijds Voorburgwal, Het Lieverdje, Spui,

Begijnhof (kapel), …

• DBpedia– Spui (Amsterdam), Universiteit van Amsterdam,

Maagdenhuis (only NL), Het Lieverdje (only NL), …

• Geonames– Nieuwezijds Voorburgwal, Lutherse Kerk,

Het Lieverdje, Spui, Begijnhof (RK), Begijnhof, ...

Merging geo info Non trivial:

Different schemas Different labeling conventions Different (GPS) data

Spui in Amsterdam has at least 5 coordinates in LOD Errors in human annotations Resolve conflicts

Begijnhof isa way Begijnhof isa area

Begijnhof isa building

“Spui” Square and the

“Lutherse Kerk”

User Interaction

• Facets={ location, event, artwork or people}• SubFacets={ painting, photograph, book, artist,

musician, politician, sport, conflict,..}

h = [0..359]

“Spui” Square and the

“Lutherse Kerk”

“Maagdenhuis” Building

and

the “Helios Building”

Technical details

Inference structure• geolocation, represented by a point s = lat, long , ⟨ ⟩

e.g. s = 52.3638611, 4.88944⟨ ⟩• Raw Mental map : spatially relevant objects (POIs)

Space package e.g. 5 different “Spui” in DBPedia, geonames, openstreetmaps and freebase)

• Aligned Mental Map: (GeoNames)i.e.: Spui square in Amsterdam

• Semantic Crawling: query cultural heritage cloud “spui+amterdam” -> construction of RDF graph

• Classify: add facets and subfacets• Choose heading h=[0..359]• Presentation

light weight client with a heavy endpoint

Demo

Copyright issues with content…

mobile DBPedia tabulator:http://bit.ly/lodapp

http://bit.ly/i-ams

Evaluation• A lot of links in LOD, actual “knowledge” rather limited:

-> A lot of comment and labels.• Alignment of non-stand URIs:

-> e.g., /Begijnhof_,Amsterdam /Spui versus /Spui_(Amsterdam)

• Local information in local languages, e.g. English wikis are limited on Amsterdam

RDF

Google maps & Panoramio

#URIs and #RDF statements for different schema’s of the location data for the Spui square

Evaluation

Discussion

• How to combine Linked Open Data and cultural heritage data?

– Geonames->mental map->PDF graph– Combination of sources achieves an added value– Still difficult to find information for the “active tourists”

• How to present the results in a manageable way on a mobile device?

– Not “one” user: Understand task, Location awareness (GPS + heading)– Syntactic sugar– Novel user interface -> augmented television

Thank you

Previews available on request: send your UUIDchris@vanaart.com

Checkout:http://bit.ly/lodapphttp://bit.ly/i-ams

http://bit.ly/ekaw2010

This work has been partially supported by:EU project NoTube (ICT-231761), Poseidon project (supported by the Dutch Ministry of Economic Affairs under the BSIK03021 program) andAgora project (funded by NWO in the CATCH programme, grant 640.004.801).

Don’t forget to book a week-end in Amsterdam