evolving the web into a global database - advances and applications

Bizer: Evolving the Web into a global Database – Advances and Applications, 30.1.2014 Slide 1

Prof. Dr. Christian Bizer

Evolving the Web into a global Database

- Advances and Applications -


Data and Web Science Group @ University of Mannheim

3 Professors• Prof. Dr. Heiner Stuckenschmidt

• Prof. Dr. Simone Paolo Ponzetto

• Prof. Dr. Christian Bizer

5 Post-Doctoral Researchers

18 PhD Students

http://dws.informatik.uni-mannheim.de/

1. Research methods for integrating and mining large amounts of heterogeneous information within enterprise and open Web contexts.

2. Empirically analyze the content and structure of the Web.


Querying the classic Web

DBHTML


Long standing Goal

Query the Web like a single,

global database


2001 Article: The Semantic Web

Envisions three things to happen:

1.people publish data in structured form in addition to HTML pages on the Web

2.common vocabularies / ontologies are used to represent data

3.people implement cool applications that do smart things with the available data.

Tim Berners-Lee, James Hendler and Ora Lassila: The Semantic Web. Scientific American, May 2001.


13 Years Later

There are 1.3 million publications about the Semantic Web on Google Scholar, but

1. Do people publish structured data on the Web?

2. Do people agree on common vocabularies / ontologies?

3. What are the cool applications that exploit the data?


Outline

1. Linked Data

2. HTML-embedded Data

3. The Role of Wikipedia

4. Conclusions


1. Linked Data

B C

RDF

RDFlink

A D E

RDFlinks

RDFlinks

RDFlinks

RDF

RDF

RDF

RDF

RDF RDF

RDF

RDF

RDF

• by using RDF to publish structured data on the Web

• by setting links between data items within different data sources.

Set of best practices for publishing structured data on the Web in the form of a single global data graph.


Global Identifiers and Links as Integration Hints

publishing Identity Links on the Web

publishing Vocabulary Links on the Web

<http://www4.wiwiss.fu-berlin.de/is-group/resource/persons/Person4> owl:sameAs <http://dblp.l3s.de/d2r/resource/authors/Christian_Bizer> .

<http://xmlns.com/foaf/0.1/Person> owl:equivalentClass <http://dbpedia.org/ontology/Person> .


Effort Distribution between Publisher and Consumer

Publishers or third parties provides

identity/vocabulary links

Consumer mines missing identity/vocabulary links

Effort Distribution


W3C Linking Open Data Project

Grassroots community effort started in 2007 to• publish existing open license datasets as Linked Data on the Web• interlink things between different data sources• maintain a data set catalog on the CKAN DataHub


LOD Datasets on the Web: September 2011

295 data sets 31,6 billion RDF triples 503 million RDF links


Newer statistics LODstats (University of Leipzig, 2014): 928 data sets LDspider Crawl (University of Mannheim, 2013): 850 data sets

Distribution by Topical Domain (September 2011)

Domain Data Sets Triples Percent RDF Links Percent

Media 25 1,841,852,061 5.82 % 50,440,705 10.01 %Geographic 31 6,145,532,484 19.43 % 35,812,328 7.11 %

Government 49 13,315,009,400 42.09 % 19,343,519 3.84 %Library 87 2,950,720,693 9.33 % 139,925,218 27.76 %

Cross-domain 41 4,184,635,715 13.23 % 63,183,065 12.54 %Life sciences 41 3,036,336,004 9.60 % 191,844,090 38.06 %User content 20 134,127,413 0.42 % 3,449,143 0.68 %

SUM 295 31,634,213,770 503,998,829


Ontological Agreement

Out of the 295 data sources• 102 (35%) only use terms from common vocabularies

• 105 (36%) only use proprietary terms

• 88 (29%) mix common and proprietary terms

Popular Vocabularies

Vocabulary # Data SetsDublin Core 92 (31.19 %)FOAF 81 (27.46 %)SKOS 58 (19.66 %)GEO 25 (8.47 %)AKT 17 (5.76 %)BIBO 14 (4.75 %)Music Ontology 13 (4.41 %)SIOC 10 (3.39 %)


Uptake in the Government Domain

Goals• Make data available to the public and other government agencies

• Ease data integration by providing unique identifiers and by setting links

W3C Government Linked Data Working Group


Uptake in the Libraries Community

Institutions publishing Linked Data• Library of Congress (subject headings)

• German National Library (PND dataset and subject headings)

• Swedish National Library (Libris - catalog)

• Hungarian National Library (OPAC and Digital Library)

• Europeana Digital Library (4 million artifacts)

Goals: 1. Integrate Library Catalogs on global scale

2. Interconnect resources between repositories (by topic, by location, by historical period, by ...)


Industry Uptake

Media Industry• British Broadcasting Corporation

• New York Times

• Wolters Kluwer

• Springer

Pharmaceutical Industry• Johnson & Johnson

• Eli Lilly and Company

• AstraZeneca

IT Industry• IBM


2. HTML-embedded Data

Microformats

Microdata

RDFa

Websites semantically markup the content of their HTML pages using:


Schema.org

ask site owners since 2011 to markup data to enrich search results.

200+ Types: Event, Organization, Person, Place, Product, Review Encoding: Microdata or alternatively RDFa


Open Graph Protocol

allows site owners to determine how entities are described in Facebook

relies on RDFa for encoding data in HTML pages

available since April 2010


The Common Crawl


The WebDataCommons.org Project

extracts all Microformat, Microdata, RDFa data from the Common Crawl

analyzes and provides the extracted data for download

Two extractions runs• 2009/2010 CC Corpus: 2.5 billion HTML pages 5.1 billion RDF triples

• 2012 CC Corpus: 3.0 billion HTML pages 7.3 billion RDF triples

used 100 machines on Amazon EC2 • approx. 3000 machine/hours

(spot instances of type c1.xlarge) 550 EUR

Jointed effort in the context of the EU project


Websites providing Structured Data (2012)

2.29 million websites (PLDs) out of 40 million provide Microformat, Microdata or RDFa data (5.65%)

369 million of the 3 billion pages contain Microformat, Microdata or RDFa data (12.3%)

Google, October 2013: 15% of all websites provide structured data.


Breakdown by Encoding Format and Site Popularity

Grouped by Alexa Website Popularity Rank (rank based on amount of page views)


Top Classes:

Topics• CMS and Blog

metadata

• Product data

• Ratings/Reviews

• Company listings

RDFa Topics (CC 2012)

og = Facebook‘s Open Graph Protocol


Top Classes:

Topics• CMS and Blog

metadata

• Navigationalmetadata

• Products and offers

• Business listings

• Ratings

• Places

• Events

Microdata Topics (CC 2012)

schema = Schema.orgdatavoc = Google‘s Rich Snippet Vocabulary


Class / Property Distribution

A small set ofclasses / propertiesis used.

Strong focus onSchema.org andFacebook vocabularies


Looking Deeper into the E-Commerce Data

Microdata(2012)

Example Names:• AppleMacBook Air MC968/A 11.6-Inch Laptop• Apple MacBook Air 11-in, Intel Core i5 1.60GHz, 64 GB, Lion 10.7

Example Description:• Faster Flash Storage with 64 GB Solid State Drive and USB 3.0 …


Usage of Schema.org Data @ Google

Rich snippetswithin

search results


Usage of Open Graph Protocol Data @ Facebook

allows site owners to determine how entities are described in Facebook


Valuable Resource for Comparison Shopping Sites

We analyzed 1.9 million product offers from 9200 shops We trained classifier for 9 product categories on product descriptions

from Amazon.


Identity Resolution for Electronic Products

We trained parser for product descriptions on offers for electronic products from Amazon.

We used Silk framework for identity resolution.


Linked Data vs. HTML-embeded Data

LOD Cloud Microdata, Microformats, RDFa< 1000 sources millions of sources

covers wider range of specific topics focused on search engines and Facebook

contains more complex data structures

very simple and shallow data structures

partial ontology agreement strong ontology agreement

data integration eased by RDF links data integration requires NLP techniques


Title

Description

CrossLanguageLinks

Geo-Coordinates

Images

Infoboxes

3. The Role of Wikipedia


Extracting Knowledge from Wikipedia


The DBpedia Knowledge Base - Version 3.9

describes 4.00 million things, out of which 3.22 million are classified in a consistent ontology using 529 classes and 2217 different properties• 832,000 persons • 639,000 places • 209,000 organizations • 116,000 music albums

Altogether 2.46 billion pieces of information (RDF triples)• 24,000,000 links to external web pages• 27,200,000 external links into other RDF datasets

DBpedia Internationalization• provide data from 119 Wikipedia language editions for download• 24 popular languages we provide cleaned infobox data


http://www.land-der-ideen.de/de


1. Answer fact queries: “birthdate michael douglas”

2. Compare things: „compare eiffel tower vs empire state building”

Applications of Google‘s Knowledge Graph


Applications of Google‘s Knowledge Graph

3. Enrich search results with infoboxes and lists• Infoboxes might also contain Microdata/RDFa data, e.g. concerts of a band

4. Rank of search results using new Hummingbird ranking algorithm


DBpedia as Background Knowledge for Data Mining

Which factors correlate with unemployment in France?


Unemployment Table with additional Attributes


RapidMiner Linked Open Data Extension

Allows you to 1. link local table to DBpedia and other LOD data sources

2. extend local table with additional attributes

3. mine extended tables using all Rapidminer features


Finding Correlations

Use additional attributes to find interesting correlations

Example correlation for unemployment in France:• African islands, Islands in the Indian Ocean,

Outermost regions of the EU (positive)

• Population growth (positive)

• Disposable income (negative)

• Energy consumption (negative)

• Fast food restaurants (positive)

• Hospital beds/inhabitants (negative)

• Police stations (positive)


Conclusions

1. Publication of Structured Data• There is more data than most people from research and industry like

• Exciting test-bed for data profiling and data integration techniques

• Not even the research focus has moved to the integration of 1000s of sources

2. Ontological Agreement• Application-pull helps (Google et al.)

• But data source-specific attributes are also important (e.g. in life science or statistics domain)

3. Applications• the big players are moving

• there is a lot of experimentation in industry, but many efforts are still in the prototype stage


Thanks

Mannheim Linked Open Data MeetupFree beer and food

Talks by Springer, Wolters Kluwer, Semantic Web Company, LOD2 project participants, DWS group members

Sunday, February 23, 2014, 6:30 PM

http://www.meetup.com/OpenKnowledgeFoundation/Mannheim-DE/1092882/

Advertisement

evolving the web into a global database - advances and applications

Internet

global database

enrich search

topics cms

site owners

linked data

rdfa data

subject headings

electronic