assessing linked data mappings using network measures

ESWC - May 2012 Assessing Linked Data mappings 1/25

Assessing Linked Data Mappings using Network Measures

Christophe Guéret, Paul Groth, Claus Stadler, Jens Lehmann

9th Extended Semantic Web Conference (ESWC)May 29, 2012

http://latc-project.eu http://www.vu.nlhttp://aksw.org

http://latc-project.eu/

http://www.vu.nl/


The next 25+5 minutes

The impact of links in the Web of Data

Main questions

What is the impact of link creation?

Can we detect “bad” links based on their impact?

Is adding links always a good thing?

Contributions

A framework to assess the impact of links

Results for 5 metrics


Is this a good or a bad link ?


Measuring the Web of Data

Look at the topology using network analysis tools

Impossible to get the complete graph

Sampling of the graph focusing on specific nodes

See the bigger picture through aggregation

Build the local network around a resource

Repeat the process a sufficient number of time


Network sampling process

Use SPARQL end point or de-reference the resources to get the descriptions


Aggregation of local results

…

ObservedTarget


Metrics

Compute local scores for a resource

Criteria

Use only the local network

Representative of a global property

Not sensitive to change of observation scale

5 metrics currently available in LinkQA


What do we want to see?

Increase of connectivity within topical groups

Increase chances of finding related information

More bridges between topical groups

Improve browsing capabilities

More connectivity around hubs

Decrease the dependency upon the hubs


Metric 1 – Degree

Metric

Number of edges around the target node

Target

Power-law distribution of values

Intuition

Presence of hubs


Metric 2 – Clustering coefficient

Metric

Density of links around the target node

Target

Increase clustering around nodes

Intuition

Topical clusters


Metric 3 – Centrality

Metric

Ratio between outgoing and incoming links

Target

Lower the discrepancy between the values

Intuition

Hubs are sensitive


Metric 4 – SameAs chains

Metric

Number of “open” sameAs chains

Target

No open sameAs

Intuition

Peer agreement


Metric 5 – Description enrichment

Metric

Richness of resource description

Target

Increase as possible

Intuition

“SameAsed” resources are complementary


Under the hood of LinkQA

http://www.flickr.com/photos/cradlehall/5747161514


Workflow of an analysis


Output of an analysis

Results on the node and aggregated scale

Per metric:

Indication of change with respect to the target

Sorted list of outlier nodes, sorted by their distance to the target

Plus, a global ranking of nodes

=> Input for manual inspection by an expert


Experimental results


Global impact of links

Observe the distributions to detect bad links


First evaluation

160 linking specifications for Silk, developed in the context of LATC

6 linking specifications with manual verification of results

50 positive links

50 negative links

Execute LinkQA with 10 samples of 50 links


Results of the detection

“C” if change detected in > 50% of runs


Some explanations

Low sensitivity of metrics:

Lack of data

Stable change

50/50 accuracy of detection:

Targets may not be the right ones

Sample may not be big enough

Semantics agnostic measures are less performant


A closer look at the outliers

See if the outliers are necessarily bad links


Second evaluation

Linking specifications for Silk, developed in the context of LATC

All linking specifications sampled to have

45 positive links

5 negative links

Execute LinkQA five time, on five samples


Rank of positive and negative links


Take home message

LinkQA is a node centric approach to measure the impact of links in the WoD network

Scalable, can be distributed

Current results show that

The 5 metrics defines are to be improved

Metrics considering Semantics perform better

The network sample seems too small

Outliers detection improves with the number of metrics

assessing linked data mappings using network measures

Technology

impact

eswc

change

links

web

developed

hubseswc

www