creating semantic fingerprints for web documents

100.00.2009 OVGU Präsentation

/22Katrin Krieger – Creating Semantic Fingerprints for Web Documents 1

Creating Semantic Fingerprints for Web Resources

Katrin Krieger, Jens Schneider, Christian Nywelt, Dietmar RösnerOtto-von-Guericke Universität Magdeburg (Germany)



Motivation

• Automatic extraction of information and generating formal

semantic descriptions are important aspects of Semantic Web

research

query

compare

combine

http://mehmetveysiadam.com



Semantic Fingerprints (SF)

• Semantic signatures of Web documents

• Representing concepts to be found in documents as well as

relationships between these concepts

• Graph structures with concepts as nodes and relationships as

edges

• Can be used to compute semantic relatedness, e.g. in e-learning

scenarios



Desired Properties of Semantic Fingerprints

P1 Concepts are distinct and unambiguous

P2 Concepts are connected through relationships

P3 Documents with similar content will

yield similar SF

P4 A SF covers all essential concepts

belonging to a document



General Idea

• Hypothesis: semantically related concepts of a domain are

connected through relationships

• This information is inherent in LOD datasets which we can exploit

to disambiguate concepts

• This information is sufficient to build semantic fingerprints



How to automatically obtain a Semantic Fingerprint

1. Extract keywords from Web document

2. Create nodes by mapping keywords to semantic concepts

3. Add edges by finding relations

4. Remove irrelevant nodes and edges

5. Identify all connected subgraphs

6. Choose semantic fingerprint from connected subgraphs



Extracting Keywords and Mapping to Concepts

• Use Natural Language Processing (NLP) tools to extract nouns and

noun phrases

• Query dataset to find concepts whose labels correspond with

keywords



Result of step #1

Disconnected graph with n concepts per keyword



Find relationships

• Expand each node and search for neighboring concepts to “grow”

the graph (BFS) up to a certain path length n



Result of Step #2

• Graph with connected subgraphs



Removing irrelevant nodes and edges

Which nodes and edges are really relevant for the semantic

fingerprint?

Heuristics:

• Path length

• Number of connecting paths

• Occurences in paths

• Number of corresponding keywords

• Interconnection property



Identifying subgraphs and picking the SF

• Identify subgraphs by performing BFS

• Determine which of the subgraphs is the semantic fingerprint

• Cover as many keywords as possible

• Number of concepts in the subgraph



Evaluation

P1 Concepts are distinct and unambiguous

P2 Concepts are connected through relationships

P3 Documents with similar content will

yield similar SF

P4 A SF covers all essential concepts

belonging to a document



Quantitative Evaluation

• P3: Documents with similar content will yield similar SF

• Extraction of 11 different KW lists from real world e-learning

documents

• Generation of SF for all KW lists

• Generation of SF for all (|KWi| k)− -tuple subsets for each KWi with |KWi|

denoting the number of keywords in KWi and varied k from 1 to 4

• Comparison of SF of original KW lists with varied KW lists

• Number of contained concepts

• Number of common concepts



Quantitative Evaluation (2)

● Number of concepts in 1992 SF vary

from 0 to 22● SF with 14-16 concepts make up 8.3%● SF with 10-13 concepts make up 20.8%

● Grouping into bins● Majority of SF with one KW

less still have ≥90% KW in

common with original SF



Quantitative Evaluation

• P1: Concepts are distinct and unambiguous

• P4: A SF covers all essential concepts belonging to a document

• Evaluation with human reviewers:

• the reviewers rated the behavior of our algorithm as comprehensible and

the fingerprints as suitable for the keyword lists

• The reviewers also found that some concepts seem to be more important

than others.



Conclusion

• New method to create a formal semantic description of

a document

• Exploits inherent properties and structures in LOD

datasets

• No need for other methods such as statistics

Open Issues

• Runtime is rather high and expensive in computing

resources

• Not all semantic relations from the documents are also

in the dataset

• Scalability



Outlook

• Exploit DOM structure of the document

• Add weights to keywords

• Investigate other data structures and adapted expansion

algorithms

• Study other methods to capture semantic relationships from text



Thank you for your attention.

What are your questions?

img src: https://flic.kr/p/6DBVxb

[email protected]



SF for KW ={“haskell”, “fold”, “higher order function”, “prove”}



In use SlideshareConnector

SlideshareConnector

StackOverflowConnector

StackOverflowConnector

FreebaseConnectorFreebaseConnector

DBpediaConnectorDBpedia

Connector

LectureSlideConnector

LectureSlideConnector

Educational metadata

Educational metadata

RESTbasedWeb-Service

(Codename: Guinan)

creating semantic fingerprints for web documents

Science