Woogle4MediaWiki Social search and knowledge sharing for MediaWiki

Hans-Jörg Happel, FZI2010-05-03 @Ontoprise Halo Team

Agenda• Motivation• Features / Demo• Technical Details• Outlook & Discussion

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 2

About me• Working as a researcher at FZI/WIM

since 2005• Core Topics:

– Knowledge Management, Software Engineering, Collaboration, Semantic Wikis

• Currently wrapping up PhD Thesis– Woogle4MediaWiki is one of the results– This Wednesday@AIFB

• Graduiertenkolloquium Angewandte Informatik „Bedarfsgetriebener Wissensaustausch in verteilten Teams“; 05. Mai 2010, 15:45 Uhr

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 3

MOTIVATION

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 4

Wikis• Web-based repositories for sharing

consensual knowledge in a collaborative fashion

• In enterprises, Wikis often act as a layer of "glue" code relating content from other information systems and capturing all the rest– Wikis are a typical entry point for information

seeking– People are often confused why and what to

put into the Wiki

5Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009

There are two kinds of people in a Wiki…

Readers• Have information needs

– But only few means for articula-ting such needs (e.g. search)

• Have collaboration needs– Search is an inherently social

activity (40,4% of users interact with others before and during search; 58,7% share information after search [EC08])

Contributors• Have limited ressources

(time/effort) and face futher barriers (motivation, privacy)

• Choices on contributions are seldom based on demand of readers (lack of guidance)

• Wiki communities stress the relevance of content („what is worth to be documented“)

6

• Knowledge sharing is a communication process• Artificial decoupling of readers and contributors in Wikis• Only few readers become contributorsSocial Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009

• Search in MediaWiki– No query logs– „Red links“ (links to non-existing pages) are

the basic means to „demand“ content– No collaboration

7Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009

Contributing in MediaWiki

• Are Wikis really contribution-friendly?– „Red links“ do not tell how much this

information is required– The „empty-page“ phenomenon (How to

start an article)

8Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009

(Media)Wikis can do better• Improve search with collaborative

features• Allow people to express information

needs• Provide a first-order representation for

information needs• Guide contributions by aggregating

information needs („need-driven knowledge sharing“)

9Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009

Need-driven Knowledge Sharing (NKS)

10

Collect information needs (e.g. from query logs)

Collect information needs (e.g. from query logs)

Calculate aggegate unsatisfiedinformation needs

Calculate aggegate unsatisfiedinformation needs

Identify private files worth sharing („Inverse Search“

[Hap08b])

Combine Wikis and Search („Woogle“)

Combine Wikis and Search („Woogle“)

FEATURES / DEMO

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 11

Searching MediaWiki and beyond…• Can replace MediaWiki built-in search or

act as an additional search engine (user configurable!)

• Two flavours– WoogleNative: provides Lucene search

features in plain PHP– WoogleRemote: uses backend which

supports

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 12

Every query can be a Wiki page…• Allows for

– Wiki-Text („Social Did-You-Mean“)

– Discussion page– Watches/Notifications

(WoogleAlerts)

• Shows basic statistics about query– „How many people

searched for that?“

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 13

Social Search• Add, remove and re-rank query results• Add comments

• Beyond that: – How to support truly collaborative information seeking (for

researchers, lawyers etc!)?

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 14

Demand guidance• Woogle uses queries and further metadata to qualify desired

content• Users are informed how strong the information need is, and in

which context it arises

15Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009

Statistics

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 16

Woogle Configuration

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 17

TECHNICAL DETAILS

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 18

WoogleNative• First MediaWiki extension for running on

Zend Lucene Search (= Java Lucene Port for PHP)

• Performance experiences:– Can handle Wikis up to 50.000 pages

(Wikipedia size) and serves normal queries in >1 sec

– Worse for Wildcard queries etc.

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 19

WoogleRemote• Connects to TeamWeaverIS via webservices

– Connecting different search engines generally possible

• TeamWeaverIS– Yet another Lucene-based search framework– Supports Query-Logs-As-A-Service

(Anonymized, aggregate query & click logging)– Comes with crawlers for File shares, Web, SVN,

CVS, Bugzilla, JIRA, JDBC, Serveral Wikis– Comes with search UIs & configuration files

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 20

Aggregate query & click logging

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 21

Implementation Trivia• Search UI

– Completely intercepts MW search process – indenpendent of MW search interfaces

• Red Links– Realized via JavaScript/jQuery (parses red links, retrieves needs from

backend batch-wise) to avoid patching MW

• Very basic implementation of access rights (via MediaWiki groups)

• Instrumentation addon– Allows to re-configure most system features based on user groups– Provides evaluation participation dialog– Logs various user behaviour data

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 22

docdoc

twIS Backend(Public Index)twIS Backend(Public Index)

docdoc

docdoc

docdoc

User 2

docdoc

docdoc

docdoc

User 1

twIS EclipsetwIS Eclipse twIS Eclipse RCPtwIS Eclipse RCP

JSF FrontendJSF Frontend

Web BrowserWeb Browser

WoogleWoogle

twIS Backend 2twIS Backend 2

twIS Backend 3twIS Backend 3

DirectAuthDirectAuthAuth via

WikiAuth via

Wiki

Push IndexerPush Indexer

DirectAuthDirectAuth

Search, Crawl

Search, Crawl

Push IndexPush Index

CrawlCrawl

SearchSearch

Big Picture

TeamWeaver Integrated SearchSimple Lucene-based toolkit for searching (crawling + Web

and Eclipse frontends) different data sources (e.g. files, SVN, CVS, Ticket Systems, Wikis); plus provides DB-based query

logging and partially semantic features (metadata extraction, ontology-based working context model in Eclipse)

TeamWeaver Integrated SearchSimple Lucene-based toolkit for searching (crawling + Web

and Eclipse frontends) different data sources (e.g. files, SVN, CVS, Ticket Systems, Wikis); plus provides DB-based query

logging and partially semantic features (metadata extraction, ontology-based working context model in Eclipse)

Inverse Search is an Eclipse-based Desktop search tool for selecting local/private documents for sharing based on a

TeamWeaver Integrated Search backend

Inverse Search is an Eclipse-based Desktop search tool for selecting local/private documents for sharing based on a

TeamWeaver Integrated Search backend

Woogle is an extension for MediaWiki, which can access a a TeamWeaver Integrated Search backend. It leverages query statistics to guide knowledge sharing in the Wiki.

Woogle has further collaboration features upcoming (e.g. Google SearchWiki-style social ranking).

Woogle is an extension for MediaWiki, which can access a a TeamWeaver Integrated Search backend. It leverages query statistics to guide knowledge sharing in the Wiki.

Woogle has further collaboration features upcoming (e.g. Google SearchWiki-style social ranking).

OUTLOOK

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 24

Outlook• Roadmap for Woogle4MediaWiki

– Lightweight integration with SMW (i.e. using semantic in result snippets)

– Social search– Request feature– Support users creating redirects

• Further (S)MW-related stuff– Semantic Need (= Woogle for SMW)– SMW4SE, SRF_UML

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 25

Summary• Woogle addresses the atificial separation of information seeking

and information provision in Wikis– Offers information seekers means to express needs– Guides contributors by aggregating demands

• Find downloads, demos, slides etc. at– http://www.mediawiki.org/wiki/Extension:Woogle4MediaWiki– http://www.teamweaver.org– http://amazonas.fzi.de/wooglenative/– Feedback & additional evaluation partners sought!

• Questions?

• Discussion– Any recommendations concerning templating/MVC?– Your experience with (UI)-Testing and Continuous Integration?– Your experience with UI surveys?– What would „Social Semantic Search“ mean to you?

26Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009

BACKUP SLIDES

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 27

28

Collaborative description of the information need

Notifications concerningsearch activities

Discussion page

Directly create new knowledge from the search dialog

Search has a URI and can be linked from within the Wiki(one page per search term)

Search across different data sources/types (WoogleRemote)

Additional clues with meta-information about a need

http://localhost/wiki/Woogle:Woogle4MediaWiki

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009

Demand guidance

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 29

Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009 30

Literature• [EC08] Evans, B. M. and Chi, E. H. 2008. Towards a model of understanding social search. In

Proceedings of the ACM 2008 Conference on Computer Supported Cooperative Work (San Diego, CA, USA, November 08 - 12, 2008). CSCW '08. ACM, New York, NY, 485-494.

• [Hap08a] Hans-Jörg Happel: Growing the Semantic Web with Inverse Semantic Search. In Proceedings of the 1st Workshop on Incentives for the Semantic Web (INSEMTIVE '08)

• [Hap08b] Hans-Jörg Happel: Closing Information Gaps with Inverse Search. In Proceeedings of the 7th International Conference on Practical Aspects of Knowledge Management (PAKM2008)

• [Hap09a] Hans-Jörg Happel: Woogle – On Why and How to Marry Wikis with Enterprise Search. In Proceedings of the 2nd Workshop on Integrated Knowledge Management Systems (IKMS2009)

• [Hap09b] Hans-Jörg Happel: Towards Need-driven Knowledge Sharing in Distributed Teams. In Proceedings of the 9th International Conference on Knowledge Management (I-KNOW 2009)

• [Hap09c] Hans-Jörg Happel: Social search and need-driven knowledge sharing in Wikis with Woogle. In Proceedings of the 5th international Symposium on Wikis and Open Collaboration (Orlando, Florida, October 25 - 27, 2009). WikiSym '09. ACM, New York, NY, 1-10.

31Social Search and Need-driven Knowledge Sharing with Woogle / WikiSym 2009