semantic search engines – on the way to web 3.0

86
A. Frank Semantic Search Engines On the Way to Web 3.0 ששששש ששששש – שששששששWeb - שששש ש3.0 ששששש שששש ששששש ששששש ששששששששש- שששששששששש שש[email protected]

Upload: hedva

Post on 09-Feb-2016

79 views

Category:

Documents


1 download

DESCRIPTION

Semantic Search Engines – On the Way to Web 3.0. מנועי חיפוש סמנטיים – Web בדרך ל-3.0. אריאל פרנק מחלקה למדעי המחשב אוניברסיטת בר-אילן [email protected]. Contents. Web 3.0 & Semantic Search General Search "Natural Language" Search Vertical Search "Social Networking" Search - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Semantic Search Engines –  On the Way to Web 3.0

A. Frank

Semantic Search Engines – On the Way to Web 3.0

– מנועי חיפוש סמנטייםWeb -3.0בדרך ל

אריאל פרנקמחלקה למדעי המחשב

אוניברסיטת בר-אילן[email protected]

Page 2: Semantic Search Engines –  On the Way to Web 3.0

A. Frank2

Contents

• Web 3.0 & Semantic Search• General Search • "Natural Language" Search• Vertical Search• "Social Networking" Search• Personalized Search

Page 4: Semantic Search Engines –  On the Way to Web 3.0

A. Frank4

“The good, the bad and the”…

Page 5: Semantic Search Engines –  On the Way to Web 3.0

A. Frank5

Web 1.0, Web 2.0, Web 3.0, Web X.0…

Page 6: Semantic Search Engines –  On the Way to Web 3.0

A. Frank6

Semantic Search

• Syntactic search – can match the query against – index of the textual content of the resources– URIs (URLs, URNs) in the system – literals in the RDF metadata– or a combination of these, possibly using:

• Exact, prefix or substring match, stemming, minimal edit distance

• Semantic search – in addition to syntactic search, can use– index of the meaning of sentences in each resource – semantic information and analysis– the graph structure of RDF metadata– or a combination of these, possibly using:

• query expansion, classification/categorization, tagging, graph traversal, microformats, RDF & OWL inferencing and reasoning

Page 7: Semantic Search Engines –  On the Way to Web 3.0

A. Frank7

Can Semantic SEs answer this)?-:

Page 8: Semantic Search Engines –  On the Way to Web 3.0

A. Frank8

Types/Examples of Semantic SEs

• General Search– MetaWeb Freebase, Yahoo! Microsearch, …

• "Natural Language" Search– Powerset, Hakia, AskMeNow AskWiki, …

• Vertical Search– Kango, AdaptiveBlue, ReportLinker, …

• "Social Networking" Search– SemantiNet, Delver, Google Social Graph API, …

• Personalized Search– Twine, MavinIT PSS, …

Page 9: Semantic Search Engines –  On the Way to Web 3.0

A. Frank9

Contents

• Web 3.0 & Semantic Search• General Search • "Natural Language" Search• Vertical Search• "Social Networking" Search• Personalized Search

Page 10: Semantic Search Engines –  On the Way to Web 3.0

A. Frank10

MetaWeb Technologies - Freebase

• Based in San Francisco, MetaWeb Technologies was spun out of Applied Minds in July 2005.

• Goal: build a better infrastructure for the Web application developers and publishers.

Page 11: Semantic Search Engines –  On the Way to Web 3.0

A. Frank11

Freebase Rational• Open, shared database of the world’s knowledge that

collects data from the Web to build a massive, collaboratively-edited database of cross-linked data.

• It is built by the community, for the community. • Free for anyone to query, contribute to, build

applications on top of, or integrate into their Web sites.• Focus is on organizing and managing complex data

structures by use of Semantic Web technologies.• Enables extraction of ordered knowledge out of the

information chaos that is the current Web.

Page 12: Semantic Search Engines –  On the Way to Web 3.0

A. Frank12

Freebase

Page 13: Semantic Search Engines –  On the Way to Web 3.0

A. Frank13

Freebase Repository

• Covers millions of topics in hundreds of categories.• Draws from large open repositories like Wikipedia,

MusicBrainz, and the SEC archives.• Contains structured information on many popular

topics, like movies, music, people and locations – all reconciled and freely available via an open API.

• Freebase information is supplemented by the efforts of a passionate global community of users, who are working together to add structured information on everything relevant.

Page 14: Semantic Search Engines –  On the Way to Web 3.0

A. Frank14

Domains and Types

Page 15: Semantic Search Engines –  On the Way to Web 3.0

A. Frank15

Google Company

Page 16: Semantic Search Engines –  On the Way to Web 3.0

A. Frank16

Freebase Help Center

Page 17: Semantic Search Engines –  On the Way to Web 3.0

A. Frank17

Freebase Semantics• Freebase spans domains, but requires that a particular

topic exist only once, even if it might normally be found in multiple databases.

• For example, Arnold Schwarzenegger would appear in a movie database as an actor, a political database as a governor and a bodybuilder database as a Mr. Universe.

• In Freebase, there is only one topic for Arnold Schwarzenegger, with all three facets of his public persona brought together.

• The unified topic acts as an information hub, making it easy to find and contribute information about him.

Page 18: Semantic Search Engines –  On the Way to Web 3.0

A. Frank18

Arnold Schwarzenegger (1)

Page 19: Semantic Search Engines –  On the Way to Web 3.0

A. Frank19

Arnold Schwarzenegger (2)

Page 20: Semantic Search Engines –  On the Way to Web 3.0

A. Frank20

Freebase Dynamics• If the user is a developer, or just mildly technical,

Freebase offers tools that make it easy to query and integrate the data into Web applications, blogs, wikis, user pages or anything else that would benefit from an injection of structured information. 

• In addition to reconciling many facets of one topic, the underlying structure of Freebase lets the user run more complex queries.

• For example, if Freebase is asked for films starring Jennifer Connelly and actors who have appeared in Steven Spielberg movies, a list of 8 movies is given.

Page 21: Semantic Search Engines –  On the Way to Web 3.0

A. Frank21

…Films starring Jennifer Connelly

Page 22: Semantic Search Engines –  On the Way to Web 3.0

A. Frank22

Freebase vs. Wikipedia• The difference lies in the way they store information. • Wikipedia arranges information in the form of articles.• Freebase lists facts and statistics. Its list form is good

not only for people who like to glance at facts, but also for people who want to use the data to build other Web sites and software. (Information in an article form can’t be reused in the same way.)

• Topics covered by Freebase include subjects that are too obscure for Wikipedia, which strives for notability appropriate to an encyclopedia.  

Page 23: Semantic Search Engines –  On the Way to Web 3.0

A. Frank23

Contents

• Web 3.0 & Semantic Search• General Search • "Natural Language" Search• Vertical Search• "Social Networking" Search• Personalized Search

Page 24: Semantic Search Engines –  On the Way to Web 3.0

A. Frank24

Powerset• Powerset is a Silicon Valley

company.• Goal: build a transformative

consumer search engine based on Natural Language Processing (NLP).

Page 25: Semantic Search Engines –  On the Way to Web 3.0

A. Frank25

Powerset Rational• Unlike conventional search engines that use keywords,

Powerset reads and understands every sentence on a Webpage and allows asking questions in plain English.

• Unique innovations in search are rooted in breakthrough technologies that take advantage of the structure and nuances of natural language.

• Using these advanced techniques, Powerset is building a large-scale search engine that breaks the confines of keyword search.

• By making search more natural and intuitive, Powerset is fundamentally changing how we search the Web, by delivering higher quality results.

Page 26: Semantic Search Engines –  On the Way to Web 3.0

A. Frank26

Who proved Fermat’s last theorem?

Page 27: Semantic Search Engines –  On the Way to Web 3.0

A. Frank27

What did Steve Jobs say about the iPod?

Page 28: Semantic Search Engines –  On the Way to Web 3.0

A. Frank28

What did Bush say about Gore?

Page 29: Semantic Search Engines –  On the Way to Web 3.0

A. Frank29

Powerlabs

• Powerlabs is a community where users can:– interact with demonstrations of Powerset’s

technology before search engine launches in 2008– give feedback to help improve the "Natural

Language" indexing– suggest ideas for the ideal search engine.

• Utilizes the participation of users on such a scale and at such an early stage of development, as a recognition of the potential of crowds wisdom to guide Powerset.

Page 30: Semantic Search Engines –  On the Way to Web 3.0

A. Frank30

Powerlabs Sign In

Page 31: Semantic Search Engines –  On the Way to Web 3.0

A. Frank31

Wiki Search Sneak Peek• Access to first open search box covering Wikipedia.• Powerset uses linguistic analyses of both the query

and Wikipedia to find the best matches. • The Miniviewer allows to view highlighted matches

in the context of a Wikipedia article without ever having to leave the results page.

• By incorporating semantic information from Powerset’s indexing process into republished Wiki pages, internal page search enables a whole new kind of search: semantic-search-within-the-page.

Page 32: Semantic Search Engines –  On the Way to Web 3.0

A. Frank32

Explore Wikipedia

Page 33: Semantic Search Engines –  On the Way to Web 3.0

A. Frank33

Google acquire something

Page 34: Semantic Search Engines –  On the Way to Web 3.0

A. Frank34

Google acquire company

Page 35: Semantic Search Engines –  On the Way to Web 3.0

A. Frank35

Search Wikipedia

Page 36: Semantic Search Engines –  On the Way to Web 3.0

A. Frank36

Companies acquired in 2001

Page 37: Semantic Search Engines –  On the Way to Web 3.0

A. Frank37

Powerset PowerMouse

• PowerMouse is an application that provides a view into Powerset’s technology, letting users examine how structured information is extracted from open text.

• It is not intended as a search application per se, but allows to search for and navigate through facts encoded in Powerset’s Wikipedia index.

• It allows to see in dramatic fashion how compactly large amounts of data can be organized and displayed based on a few semantic relationships.

Page 38: Semantic Search Engines –  On the Way to Web 3.0

A. Frank38

PowerMouse Examples

Page 39: Semantic Search Engines –  On the Way to Web 3.0

A. Frank39

Google acquire something

Page 40: Semantic Search Engines –  On the Way to Web 3.0

A. Frank40

something eats carrot

Page 41: Semantic Search Engines –  On the Way to Web 3.0

A. Frank41

person won nobel

Page 42: Semantic Search Engines –  On the Way to Web 3.0

A. Frank42

Contents

• Web 3.0 & Semantic Search• General Search • "Natural Language" Search• Vertical Search• "Social Networking" Search• Personalized Search

Page 43: Semantic Search Engines –  On the Way to Web 3.0

A. Frank43

Kango

• Vertical semantic search engine for personalized travel information.

• Goal: first step to deciding where to go, where to stay or what to do; finds the trip that is right for you.

Page 44: Semantic Search Engines –  On the Way to Web 3.0

A. Frank44

Kango Rational

• Kango indexes the collective wisdom on travel from the entire Web.

• Recommendations are based on a gestalt of voices heard in over 20 million reviews, ratings, blogs, journals, and articles collected from over a thousand sources such as Web sites, books and magazines.

• Organizes and presents the most relevant opinions and product details in a "federated" search display based on what’s known about travel preferences.

Page 45: Semantic Search Engines –  On the Way to Web 3.0

A. Frank45

Kango Repository

• Kango has scoured the Web to collect all kinds of places to go, things to do and places to stay.

• It then analyzed and organized millions of travelers' opinions to enable search based on exact travel requirements and preferences.

• Kango brings together:– more than a thousand sites– 400,000 lodging, activity and destinations options – 20 million reviews, ratings and blogs.

Page 46: Semantic Search Engines –  On the Way to Web 3.0

A. Frank46

How Kango Works

Page 47: Semantic Search Engines –  On the Way to Web 3.0

A. Frank47

Kango Semantics

• It provides many options for specifying a trip.• Kango thinks about those options in terms of

the “Long Tail“ concept to help make the trips distinct and memorable.

• It "understands" the travel lingo, so it helps make informed decisions about what best fits specific travel preferences for each user.

• Kango is creating an ontology of global travel content that includes ranking of superlatives within review sites.

Page 48: Semantic Search Engines –  On the Way to Web 3.0

A. Frank48

Lodging

Page 49: Semantic Search Engines –  On the Way to Web 3.0

A. Frank49

Things to Do

Page 50: Semantic Search Engines –  On the Way to Web 3.0

A. Frank50

Kango Dynamics

• Enables new ways of filtering through its collection to get the recommendations that are most relevant to preferences and priorities.

• Based on persons traveled with, the kind of destination looked for, and what is likely to be done, it sifts through its information to deliver the right getaway.

• For example, returns– one set of hotel and activity recommendations when

traveling to Monterey for a romantic getaway – a different set when going to Monterey with the family to

visit the aquarium and hang out on the beaches.

Page 51: Semantic Search Engines –  On the Way to Web 3.0

A. Frank51

Old Monterey Inn

Page 52: Semantic Search Engines –  On the Way to Web 3.0

A. Frank52

Campgrounds in Hawai

Page 53: Semantic Search Engines –  On the Way to Web 3.0

A. Frank53

Contents

• Web 3.0 & Semantic Search• General Search • "Natural Language" Search• Vertical Search• "Social Networking" Search• Personalized Search

Page 54: Semantic Search Engines –  On the Way to Web 3.0

A. Frank54 A. Frank

SemantiNet• SemantiNet is a startup, based

in Tel Aviv, that is creating a new revolutionary technology that is based on Semantic Web concepts.

• Goal: leverage Web information in a meaningful way to boost the manner users experience the Internet.

Page 55: Semantic Search Engines –  On the Way to Web 3.0

A. Frank55 A. Frank

SemantiNet Rational

• SemantiNet makes life easy by allowing users to take advantage of the variety and richness of information and services that exist on the Internet, but in a way that is simple, smart and intuitive.

• SemantiNet leverages Semantic Web concepts to seamlessly integrate information and services enabling users to achieve more while working less!

Page 56: Semantic Search Engines –  On the Way to Web 3.0

A. Frank56 A. Frank

SemantiNet Repository

• SemantiNet collects relevant information from common social networks and established Web sites in order to provide users with a customized and efficient personalized and contextual browsing experience.

• Relevant personal information can be – entered on their Web site – provided by users through use of SemantiNet – or extracted from "traffic data" generated by

browser use.

Page 57: Semantic Search Engines –  On the Way to Web 3.0

A. Frank57 A. Frank

SemantiNet Semantics

• Develops a semantic framework solution that allows for rapid deployment of Web mashups, applications and services, in a way that enhances the way people use the internet.

• Rather than simply aggregating information, SemantiNet’s technology, integrates information as well as mashing it as needed.

• The idea is to bring the relevant online content to the user rather than the user to the content.

Page 58: Semantic Search Engines –  On the Way to Web 3.0

A. Frank58

SemantiNet Demo

Page 59: Semantic Search Engines –  On the Way to Web 3.0

A. Frank59

SemantiNet Demo

Page 60: Semantic Search Engines –  On the Way to Web 3.0

A. Frank60

SemantiNet Demo

Page 61: Semantic Search Engines –  On the Way to Web 3.0

A. Frank61

SemantiNet Demo

Page 62: Semantic Search Engines –  On the Way to Web 3.0

A. Frank62

Example of Social Graph

Page 63: Semantic Search Engines –  On the Way to Web 3.0

A. Frank63

Delver• Delver (formerly Semingo) is

headquartered in Herzeliya and will officially open U.S. offices in Silicon Valley in spring of 2008.

• Goal: provide a semantic search engine that allows users to search for information created and referenced by their own social graph.

Page 64: Semantic Search Engines –  On the Way to Web 3.0

A. Frank64

Delver Rational• Delver provides a “connected search engine” that

allows users to find content, media and people within their network via a simple search interface.

• Delver organizes and ranks content from the user’s network because social connections are critical for discovering more personally relevant information.

• It indexes the social Web (social networks, blogs, social applications, etc.), and cross-connects the data with users’ social graph.

• Improves the relevancy of Web search results by prioritizing these results based upon the specific searcher’s social network.

Page 65: Semantic Search Engines –  On the Way to Web 3.0

A. Frank65

Delver Repository• Delver begins by crawling the Web in order to map

users’ social connections. • It specifically indexes people's social connections on

flickr, MySpace, LinkedIn, YouTube, hi5, facebook, Blogger, and more sites are being added all the time. 

• Instead of just looking at a Web site's popularity, Delver looks at information like whether your friends have tagged the site or if it's found on their social network profiles, bookmarking sites, photos and video sharing sites, or on their blogs.

• The results are more relevant because they account for who a person is and what it finds valuable.

Page 66: Semantic Search Engines –  On the Way to Web 3.0

A. Frank66

Liad Agmon

Page 67: Semantic Search Engines –  On the Way to Web 3.0

A. Frank67

Venture Funding

Page 68: Semantic Search Engines –  On the Way to Web 3.0

A. Frank68

Delver Semantics• Delver knows who a user is and who his friends are

even if users didn't import their address book or add their "Social Networking" profiles.

• Instead, Delver leverages the social graph to map out a user's social connections.

• Since everyone's social graph is unique, like a fingerprint, the same Delver query will yield significantly different results for each user – as reflected through the collective experiences of each person’s contacts.

• The results are more personal and meaningful to users than a generic search using a "normal" search engine.

Page 69: Semantic Search Engines –  On the Way to Web 3.0

A. Frank69

Delver Dynamics

• When a user performs a query, results from all over his social Web are displayed.

• Even if a user and others are not directly related as "friends" on a social network, the plus sign the beneath picture can still be clicked to add them as a connection.

• This way, a user can view the relevant bookmarks, links, blog posts, photos, and videos of people like him even if he doesn’t know them personally... and they don't have to confirm the connection on their end.

• Alternately, a user can choose to exclude certain connections from his search results.

Page 70: Semantic Search Engines –  On the Way to Web 3.0

A. Frank70

Roi Carthy

Page 71: Semantic Search Engines –  On the Way to Web 3.0

A. Frank71

Visit New York

Page 72: Semantic Search Engines –  On the Way to Web 3.0

A. Frank72

Contents

• Web 3.0 & Semantic Search• General Search • "Natural Language" Search• Vertical Search• "Social Networking" Search• Personalized Search

Page 73: Semantic Search Engines –  On the Way to Web 3.0

A. Frank73

Radar Networks Twine• Radar Networks, a pioneer of

Semantic Web technology, introduced Twine.

• Goal: enables individuals and groups to organize, share and discover information and knowledge around their interests.

Page 74: Semantic Search Engines –  On the Way to Web 3.0

A. Frank74

Twine Rational• Twine is a "knowledge networking" tool designated

as a revolutionary Semantic Web application. • It is a new service that helps organize, share and

discover information about user interests, with networks of like-minded people.

• Twine can be used alone, with friends, groups and communities, or even in a company.

• It has aspects of social networking, wikis, blogging, knowledge management systems – but its defining feature is that it's built with Semantic Web technologies.

• It aims to bring a usable and scalable interface to the long-promised dream of the Semantic Web.

Page 75: Semantic Search Engines –  On the Way to Web 3.0

A. Frank75

Twine Repository• Using Twine, a user can:

– add content via Wiki functionality (has many post types)– email content into the system – and "collect" something (as an object, e.g., a book object).

• Twine ties it all together:– As information is added to Twine, it is automatically tagged

so that it can be easily found. – Users can connect with individuals and groups, gather and

share content, and engage in discussions around interests. – Twine connects between new people, content and products

that match their interests, and also helps users discover other people and their contributions.

Page 76: Semantic Search Engines –  On the Way to Web 3.0

A. Frank76

Twine Semantics• Twine is powered by semantic understanding.• At first glance it is very much like Wikipedia, but

there is a whole lot more smarts to the system.• It's not based around socializing, but aims to share

information and automatically organize it, learn about user interests, and make varied connections and recommendations.

• The more it is used, the better it understands the user interests and the more useful it becomes.

• It is a "Semantic Graph", which maps relationships to both people and topics.

Page 77: Semantic Search Engines –  On the Way to Web 3.0

A. Frank77

Twine Sign In

Page 78: Semantic Search Engines –  On the Way to Web 3.0

A. Frank78

Twine Dynamics

• Enables user commenting and viewing of related things.

• Allows sharing of tags.• Enables import and export of user own data. • RSS feeds to track all kinds of things (topics,

events, search, etc).• Semantic Web technologies are being used:

RDF, OWL, SPARQL, XSL, GRDDL.• An open platform - there will be SPARQL and

REST APIs.

Page 79: Semantic Search Engines –  On the Way to Web 3.0

A. Frank79

Welcome Steve to Twine

Page 80: Semantic Search Engines –  On the Way to Web 3.0

A. Frank80

Explore Green Business and Investing

Page 81: Semantic Search Engines –  On the Way to Web 3.0

A. Frank81

Steve Smith’s Twine

Page 82: Semantic Search Engines –  On the Way to Web 3.0

A. Frank82

Explore Green Tech

Page 83: Semantic Search Engines –  On the Way to Web 3.0

A. Frank83

Semantically up(?-:

Page 84: Semantic Search Engines –  On the Way to Web 3.0

A. Frank84

Where does the MetaWeb fit!?

Page 85: Semantic Search Engines –  On the Way to Web 3.0

A. Frank85

References

• Web 3.0, In Wikipedia, The Free Encyclopedia, http://en.wikipedia.org/w/index.php?title=Web_3.0&oldid=123368293

• Entrepreneurs See a Web Guided by Common Sense, John Markoff , New York Times, November 12, 2006, http://www.nytimes.com/2006/11/12/business/12Web.html?ex=1320987600&en=254d697964cedc62&ei=5088

• Parts I & II: A Smarter Web, John Borland, Technology Review, March 19-20, 2007, http://www.technologyreview.com/Infotech/18396/

Page 86: Semantic Search Engines –  On the Way to Web 3.0

A. Frank86

References• M. Hildebrand, J. R. van Ossenbruggen, L. Hardman,

An Analysis of Search-based User Interaction on the Semantic Web, Report INS-E0706, May 2007, 6th Intl. Semantic Web Conference, November 2007, http://ftp.cwi.nl/CWIreports/INS/INS-E0706.pdf

• Jim Hendler, Web 3.0: Chicken Farms on the Semantic Web, IEEE Computer, January 2008, http://www.computer.org/portal/site/computer/menuitem.5d61c1d591162e4b0ef1bd108bcd45f3/index.jsp?&pName=computer_level1_article&TheCat=1075&path=computer/homepage/0108&file=Webtech.xml&xsl=article.xsl&

• Richard Waters, World-wise Web?, Financial Times, http://www.ft.com/cms/s/0/4fba0434-e98c-11dc-8365-0000779fd2ac.html?nclick