mobile cultural heritage guide: location-aware semantic search (ekaw2010)

66
Mobile cultural heritage guide location-aware semantic search Chris van Aart (1) , Bob Wielinga (1,2) , Willem Robert van Hage (1) (1) VU University Amsterdam, the Netherlands (2) Universiteit van Amsterdam, the Netherlands [email protected] October 2010

Upload: chrisvanaart

Post on 19-Jan-2015

1.219 views

Category:

Technology


1 download

DESCRIPTION

In this paper we explore the use of location aware mobile devices for searching and browsing a large number of general and cultural heritage information repositories. Based on GPS positioning we can determine a user's location and context, composed of physical nearby locations, historic events that have taken place there, artworks that were created at or inspired by those locations and artists that have lived or worked there. Based on a geolocation, the user has three levels of refinement: pointing to a specific heading and selection and facets and subfacets of cultural heritage objects. In our approach two types of knowledge are combined: general knowledge about geolocations and points of interest and specialized knowledge about a particular domain, i.e. cultural heritage. We use a number of Linked Open Data sources and a number of general sources from the cultural heritage domain (including Art and Architecture Thesaurus, Union List of Artist Names) as well as data from several Dutch cultural institutions. We show three concrete scenarios where a tourist accesses localized information on his iPhone about the current environment, events, artworks or persons, which are enriched by Linked Open Data sources. We show that Linked Open Data sources in isolation are currently too limited to provide interesting semantic information but combined with each other and with a number of other sources a really informative location-based service can be created.

TRANSCRIPT

Page 1: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Mobile cultural heritage guide location-aware semantic search

Chris van Aart(1), Bob Wielinga(1,2), Willem Robert van Hage(1)

(1) VU University Amsterdam, the Netherlands (2) Universiteit van Amsterdam, the Netherlands

[email protected] 2010

Page 2: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Data: http://bit.ly/ekaw2010

Simulator: http://bit.ly/i-ams

iOS: http://bit.ly/lodapp

Page 3: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Recovering of tourism market in

Amsterdamnot fast enough

Page 4: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 5: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 6: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 7: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 8: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 9: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 10: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 11: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 12: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 13: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 14: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 15: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 16: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

(Digital) Cultural heritage• Museums• (old)buildings (monuments)• Statues• Art-works• Historic events

Page 17: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 18: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 19: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 20: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 21: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

<title>De Prinsengracht bij de Reesluis, 1663 - 1664</title>

<keywords>schilderij, stadsgezicht, stadsgezicht Amsterdam, Prinsengracht, Reesluis, scheepvaart</keywords>

Page 22: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

<title>Roemer EERSTE STEENLEGGING NIEUWE KERK, 1647</title>

<keywords>drinkglas, roemer, kelkglas, heiplan toren Nieuwe Kerk, steenlegging, Nieuwe Kerk, Willem Backer, Cornelis Backer, Stichting Projecten De Nieuwe Kerk AmsterdamGlascollectie</keywords>

<description>Op de kelk is in diamantgravering het heiplan voor de nooit voltooide toren van de Nieuwe Kerk aangebracht.<description>

Page 23: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

<title>Anatomische les van Prof. Frederik Ruysch, 1670</title>

<keywords>schilderij, portret, groepsportret, anatomische les, ambachten en beroepen, onderwijs, Chirurgijnsgilde, Leendert Fruyt, Aert van Swieten, Frederik Ruysch, Gillis Hondekoeter, Rogier de Coen, Joris van Loon Tilanus, Jacob Brandt</keywords>

<description>Frederik Ruysch (1638-1731) was de opvolger van Jan Deijman als praelector van het chirurgijnsgilde.De jonge doctor demonstreert de ligging van de lymfevaten in de lies. De nadruk ligt nu niet op de levenloze staat van de dode, zoals bij Rembrandt. Het lichaam op de snijtafel lijkt eerder toe te behoren aan iemand die slaapt. Waarschijnlijk wilde Ruysch hiermee zijn specialiteit verbeelden: dankzij zijn preparatietechnieken leek het alsof de doden nog leefden. Het schilderij is na restauratie voor het eerst sinds lange tijd weer te zien.</description>

Page 24: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

<title>De brand in het oude stadhuis (1652)</title>

<keywords>prent, stadsgezicht, brand van het oude stadhuis op de Dam, Dam</keywords>

Page 25: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

http://bit.ly/eculture

Page 26: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Cultural heritage data cloud

http://e-culture.multimedian.nl/resources/datacloud/

Page 27: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

http://dbpedia.org/resource/Amsterdam

27

Page 28: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 29: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Location awareness / LBS• Real time locating and positioning• Sun/Stars, Compass, Postal code, WGS84 (GPS)… • communication, networks, traffic, logistics, security,

search & rescue, business administration, leisure,…

Page 30: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

DBPedia mobile: Google maps

“Louvre app”

Page 31: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Research question “How to find cultural heritage POIs at a

particular location, given the constraints of a mobile device, using current Semantic Web data and technology?”

Page 32: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Challenges1. enrich location data by constructing an

``enriched local map'' of nearby Points of Interest2. Enrich POIs with additional information, such as

e.g. events, persons and artworks.3. present the results in a manageable way on a

mobile device

Page 33: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 34: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Approach

• Combine: – general knowledge about geolocations and points of

interest:• GeoNames, LinkedGeodata, Freebase and DBPedia

– specialized knowledge about the cultural heritage domain• Art and Architecture Thesaurus, • Union List of Artist Names, • Netherlands Institute for Art History• Rijksmuseum Amsterdam.

Page 35: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

The “Active” tourist: What do I see? Next stop? Hot spots? History? Stories? Events? Persons? Descriptions? Maps?

Page 36: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

From Geolocation to Real-world Annotation

Task structure

Page 37: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

11

Page 38: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Harvest Locations Iphone GPS

Latitude = 52.368611 Longitude = 4.88944 (we ignore heading)

==> select those geo data that are relevant to this point Searching dbpedia, linkedgeodata, geonames for locations and

objects within a certain distance. Crawling via various relations (owl:sameAs, nearby etc) we gather

a collection of RDF resources related to our location. Using a Prolog spatial indexing package (space) we can compute

distances and directions from our current location.

Page 39: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

22

Page 40: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Merging geo info Non trivial:

Different schemas Different labeling conventions Different (GPS) data

Spui in Amsterdam has at least 5 coordinates in LOD There is also an Spui in The Hague

Errors in human annotations Resolve conflicts

Begijnhof isa way Begijnhof isa area

Begijnhof isa building

Page 41: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

33

Page 42: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 43: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

44

rdf:type property (a.o. restaurant, shop, building, church, place of worship, university, way, bequinage, market, marketplace)

linked geo data ontology (LGDV):Facets={ location, event, artwork or people}SubFacets{ painting, photograph, book, artist, musician, politician, sport, conflict,..}

Page 44: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 45: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

In practice

Page 46: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Spui square (52° 22 7″ N, 4° 53 22″ E′ ′ )

Page 47: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 48: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

What does LOD return?

• Linked Geo Data– Spui25, Nieuwezijds Voorburgwal, Het Lieverdje, Spui,

Begijnhof (kapel), …

• DBpedia– Spui (Amsterdam), Universiteit van Amsterdam,

Maagdenhuis (only NL), Het Lieverdje (only NL), …

• Geonames– Nieuwezijds Voorburgwal, Lutherse Kerk,

Het Lieverdje, Spui, Begijnhof (RK), Begijnhof, ...

Page 49: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Merging geo info Non trivial:

Different schemas Different labeling conventions Different (GPS) data

Spui in Amsterdam has at least 5 coordinates in LOD Errors in human annotations Resolve conflicts

Begijnhof isa way Begijnhof isa area

Begijnhof isa building

Page 50: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

“Spui” Square and the

“Lutherse Kerk”

Page 51: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

User Interaction

• Facets={ location, event, artwork or people}• SubFacets={ painting, photograph, book, artist,

musician, politician, sport, conflict,..}

h = [0..359]

Page 52: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

“Spui” Square and the

“Lutherse Kerk”

Page 53: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 54: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

“Maagdenhuis” Building

and

the “Helios Building”

Page 55: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Technical details

Page 56: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Inference structure• geolocation, represented by a point s = lat, long , ⟨ ⟩

e.g. s = 52.3638611, 4.88944⟨ ⟩• Raw Mental map : spatially relevant objects (POIs)

Space package e.g. 5 different “Spui” in DBPedia, geonames, openstreetmaps and freebase)

• Aligned Mental Map: (GeoNames)i.e.: Spui square in Amsterdam

• Semantic Crawling: query cultural heritage cloud “spui+amterdam” -> construction of RDF graph

• Classify: add facets and subfacets• Choose heading h=[0..359]• Presentation

Page 57: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

light weight client with a heavy endpoint

Page 58: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)
Page 59: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Demo

Page 60: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Copyright issues with content…

mobile DBPedia tabulator:http://bit.ly/lodapp

Page 61: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

http://bit.ly/i-ams

Page 62: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Evaluation• A lot of links in LOD, actual “knowledge” rather limited:

-> A lot of comment and labels.• Alignment of non-stand URIs:

-> e.g., /Begijnhof_,Amsterdam /Spui versus /Spui_(Amsterdam)

• Local information in local languages, e.g. English wikis are limited on Amsterdam

RDF

Page 63: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Google maps & Panoramio

Page 64: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

#URIs and #RDF statements for different schema’s of the location data for the Spui square

Evaluation

Page 65: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Discussion

• How to combine Linked Open Data and cultural heritage data?

– Geonames->mental map->PDF graph– Combination of sources achieves an added value– Still difficult to find information for the “active tourists”

• How to present the results in a manageable way on a mobile device?

– Not “one” user: Understand task, Location awareness (GPS + heading)– Syntactic sugar– Novel user interface -> augmented television

Page 66: Mobile cultural heritage guide: location-aware semantic search (EKAW2010)

Thank you

Previews available on request: send your [email protected]

Checkout:http://bit.ly/lodapphttp://bit.ly/i-ams

http://bit.ly/ekaw2010

This work has been partially supported by:EU project NoTube (ICT-231761), Poseidon project (supported by the Dutch Ministry of Economic Affairs under the BSIK03021 program) andAgora project (funded by NWO in the CATCH programme, grant 640.004.801).

Don’t forget to book a week-end in Amsterdam