Facets of Applied Semantic

MediaWiki ++SMWCon Fall 2011 Tutorial #4

2011.09.21

AboutMe: http://semantic-mediawiki.org/wiki/User:Jesse

SMW++ : Ontoprise, TeamMersion, etc.

Project Halo: http://projecthalo.com/

Vulcan Inc.Paul Allen, Idea Man

Agenda

Startup Customize

Integrate

Jumpstart

Ways to quickly start a new wiki

Using a BundleSemantic Bundle

http://www.mediawiki.org/wiki/Semantic_Bundle

Features: (Pros and Cons)A set of author-selected extensions

A fixed set of extensions, few customization options

Simplified download and configuration processStill need manual configuration

Usually up-to-date and work well togetherCommunity tested and supported

Installation Package

SMW+ Package http://smwforum.ontoprise.com/smwforum/index.php/Help:SMW%2B

Features:A set of (fixed) company selected extensions

Simplifies installation and configurationWindows installer and VM images are great

And they just became free!

You may be curious about what it does exactly…

Professionally documented and testedUsually weeks after major release of latest SMW

Deployment Framework

Really easy to use

Functionality (extensions) growing fast

Saves lots of time of maintenanceClicks rather than lots of commands

With the benefit of efficiency, quality and consistency

Only limited by the extensions in itSupported extensions and packages are growing fast

Developer Deployment

When you want a set of custom extensions, you want

Your OWN

deployment mechanism!

Deployment Practices

Your own codebase, with checkpointsSVN or Git to get the base, and then

Customize the wikiScript or Manual or Mixed

A third-party deployment framework and repository

Ontoprise: http://smwforum.ontoprise.com/smwforum/index.php/Help:Deployment_Framework

RPI: http://code.google.com/p/smwbp/wiki/setup_wiki

Referata: http://smw.referata.com/wiki/Category:Packages

Build SystemYou own it!

Your own selection of extensions

And versions of them

Own patches, scripts and templates

Vulcan: Internal GIT repositories of completed build

Versioning and branches between projects

Other examples:RPI Best Practice (Google code)

Hudson build (PNNL)

Future TrendA public wiki installation repository

with lots of packages

Including necessary data

Semantically marked up

Well documented (or even reviewed)

Easy to find and use

Federated wiki package store (Wiki AppStore)

Customize

Get your own functionality, look and feel

What to Customize

Skins

Extensions

Schema and Data

Templates and forms

Anything else you can code…

SkinsHow to customize a skin

Create your skin file in /skins folder within your wiki directory, like Ontoskin.php

Finish this skin files following Ontoskin.php skin filesThere are two classes in this skin files:

Inherit from SkinTemplate, set the CSS and template filter.

Inherit from QuickTemplate, set the UI style

Create skin folder with your skin name in /skinsCopy image and CSS files to your skin folder

Use your own skinIn LocalSettings.php set

$wgDefaultSkin = 'wiking_skin'

Seahawk Video Skin

Skin Customization

ExampleVulcan Development Wiki and Seahawk wiki.

We have customized these two skins for our projects and the key is the method of QuickTemplate. In this method, you can decide what to show and how it shows on your wiki

You can customize wiki site like menu links, page titles, CSS and JavaScript files in the data properties of QuickTemplate

For example, show wiki side bar in your skin<?php foreach ($this->data['sidebar'] as $bar => $cont) { ?> <li> <a href="#"><h5><?php $out = wfMsg( $bar );></h5></a> <ul> <?php foreach($cont as $key => $val) { ?> <li id="<?php echo $val['id'] ?>" > <a href="<?php echo $val['href'] ?>"><?php echo $val['text'] ?></a> </li> <?php } ?> </ul> </li><?php } ?>

Personalized SkinSkin supports customization via __USERNAME__

User can have personalized styles within a skin

http://www.mediawiki.org/wiki/Manual:Skin_configuration#User_CSS

Tip: A way to turn off Wikipedia banner ad:

http://en.wikipedia.org/wiki/User:Jesseone/vector.css

Schema Design

“Just Do It”

The Middle Way

Designer

Ontology

“Just Do It!”

Easy to start

Flat, large ontology, at beginning

Many properties only valid in certain cases

May have similar/duplicate items

But it’ll get better over time!

Designer Ontology

Professionally generated after rounds of discussions

Time consuming

More comprehensive and carefully designed

More data (restriction) than required

Data (ontology) importer isn’t powerful to fully support all features in the designing tool (Protégé)

The Middle WayPractical schema design

Plan and Explore

Think, adapt, and evolve

N-ary RelationsType:Record

Semantic Internal Object extension

Adding an object in the middle (Reification)

Using parameterized (patterned) properties

Using a triple store (external)

Extensions

Too many __ too few __

A few times ___

Many times ___

Every time ___

Our Story:

Searching

Data I/O

Securing

Visualizing

Customization examples

Sidebar

Template and Forms

Find ExtensionsWhere to find extensions

http://www.mediawiki.org/wiki/Extension_Matrix

By supported version (1.16, 1.17, …)

By status (beta, stable, …)

By type (API, AJAX, hook, user rights…)

By new-ness (sort by most recently created time)

By talk page new-ness…

By most updated versions…

And of course using search tools from G.Y.M…

Get the Extensions!

Some essential MediaWiki extensions!http://semantic-mediawiki.org/wiki/Help:MediaWiki_extensions

And quite a few Semantic MediaWiki extensions:http://semantic-mediawiki.org/wiki/Help:SMW_extensions

Find what you want Not exactly? Then patch it!

Nothing close? Then build it!Or find people to build it

Our ExperiencesWe built several wikis (2008-2011)

Sci-Fi movie demo

Semantic Football (Entertainment)

NGT – a work-order handling system

Ultrapedia – an Analytical Encylopedia

Wiking agile project management

Biology ontology

Stanford class wiki

Neurowiki …

Patches Often Needed

Many features are sort of there, some still not.

Popular extensions (e.g. SF, SRF, etc.) often need patches or tuning

Example: SF Auto-completion (over the years)

Fixed set of allowed values

All values in a category

All values having a specific property

All values from a custom query

New Features Added

Real-time notification (Semantic NotifyMe, 2008)

Aggregation (average, max, sum, etc.) (2009)

Video merging

Open Flash Charts (OFC) SRF (2009-2010)

Strict > and < comparison (2010)

Exhibit enhancements (webit) (2009)

New formats: Runway, Time Plot, etc.

Mapping: unique ID requirements

Geocoding

Semantic WikiTags (2009)

Semantic Connector (2010)

Two of Our Latest Extensions

Wiki Object ModelEnable third parties to consume wiki data via APIs

APIs (and URLs) for easy and flexible manipulation of wiki page (and data)

At a finer granularity

Widget Designer / Page Creator (prototype)To lower structured page creation dramatically

To enable data-driven application building

Come to my Semantic Page Creator talk on Friday

Why Wiki Object Model

Strong needs for access the structured data and unstructured data in the wiki as a data store

From internal and external applications’ point of view

A DOM-like approach is good choiceXpath is a reasonable tool and standard

Granularity offers flexibility and power

Easier to operate on known objects and calling REST APIs rather than parsing and handling wiki texts

Common gateway for both internal and external apps

Wiki Object Model Objects

Category

Link

Property

Text

Magic word

HTML tag

Sentence

Word*

Image

31

Page

Section

Parameter

Parameter value

Template

Template field

Parser function

List item

Table

Table cell

* Not yet implemented as of September 2011

Demo of WOM Demo at http://wiking.vulcan.com/dev_sandbox/ Demo 1: APIs

The API is part of MediaWiki API, with our additions to allow get and set (read/write) the content of a wiki page

http://wiking.vulcan.com/dev/index.php/Extension:Wiki_Object_Model/Apis

Demo 2: Inline Editor A simple inline editor (AJAX styled) to let user modify

a specific portion of the wiki page Based on

http://wiking.vulcan.com/dev/index.php/Extension:Wiki_Object_Model/Functions

32

Security Extensions

Many choices = nothing is perfect

Options: from very basic to very advanced:

No Security

MediaWiki Built-in Groups

Simple Security (Page Security)

HaloACL

Featu

res

Restrictio

ns

A Short CaseProposal Management System (Wiki)

Users (including privileged users) submit proposals

Reviewers (privileged users) rate and review them

Reviewers discuss and make decision and assign

What security extension should it employ?

HaloACL? – the most powerful ACL out there

SimpleSecurity? – simple and effective

Or MediaWiki built-in groups? – simplest

MediaWiki Groups

VisualizationsSemantic Result Formatter

Format = ???{Param} = ???

Visualization: Our Path

2008: SRF used to only have few options

2009: Exhibit seemed coolPorted webit

Fixed some bugs and enhanced features Runway, TimePlot, multiple rows in Map view etc.

2010: Open Flash ChartMore bars, pies, lines, scatter plot…

Interaction

Richer information display

2011: TabbedHighChart, OFC/ScatterPlot

Comparing SRF Options

Google Charts (pie and bar)Google does it! (G gets your data, no offline choice…)

Static bar and pie, no color options…

ExhibitMore choices (timeline, map, facet, runway…)

Size limit; Browser compatibility issues

Flash-based: OFC (Open Flash Chart), High Chart

Many bars, charts, plots, even scattered plots

Flash required

Sidebar Customization

Basic: MediaWiki:Sidebar

Enhanced: semantic queries, and tree views

Advanced: use variables together with queries

Custom: Using an extension: DynamicSidebar Customize your own sidebar at User:<username>/Sidebar

http://www.mediawiki.org/wiki/Extension:DynamicSidebar

Example: http://wiking.vulcan.com/dev/

Current Sprint in Sidebar

*Current sprint{{#ask: [[Category:Project sprints]][[Sprint start date::<{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]][[Sprint end date::>{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]|format=template|template=Sidebar query template|link=none|}}

My Active Sprint Tasks in Sidebar

*My tasks{{#ask: [[Category:Project tasks]][[Project task owner::<q>[[User alias::{{CURRENTUSER}}]]</q>]][[Project story::<q>[[Project sprint::<q>[[Category:Project sprints]][[Sprint start date::<{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]][[Sprint end date::>{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]</q>]]</q>]][[Project task status::In progress||Not started]]|format=template|template=Sidebar query template|link=none|}}

More Extension Examples

Twitter Feedhttp://www.mediawikiwidgets.org/Twitter_Search (example)

Flickr Feedhttp://www.mediawikiwidgets.org/Flickr

Amazon Carouselhttps://widgets.amazon.com/Amazon-Carousel-Widget/

Google Analyticshttp://www.mediawiki.org/wiki/Extension:Google_Analytics

Optify Analyticshttp://www.optify.net/

Templates and Forms

MediaWiki template is essential for its structured data handling

Editing, especially Template editing, is not trivial

Semantic Forms extension greatly lowers the bar

Customizing templates and forms can provide very visually appealing pages

Form with a Style

http://www.thethirdturn.com/w/index.php?title=Form:Driver&action=edit

Forms with autocompletion

Basic Auto-Completion on Category Values

Advanced Auto-Completion on Customized Query Results

Integration

Guidelines and case studies on building wikis

StepsKnow what to build

Customers, customers, customers!

Get your collection of extensions…

Customize your wiki (template, forms)

Handle the data (schema, query, API, …)

Case 1. Proper Wiki

Focus on Skin, Form and Template

Steps:1. Pick a (default) skin

2. Design some forms1. And templates (queries) behind the forms

3. Build starter pages and Go!

Biology Ontology Wiki

People need discuss about the details related to construct a biology ontology

Classes (categories)

Properties

Relations

Textbook analysis

Discussion

Forms necessary to keep content tidy

Ex: Property Form

Forms on Namespace

http://www.mediawiki.org/wiki/Extension:Semantic_Forms#Based_on_namespace

Case 2: Simple Workflow App

Idea: go from one form to another form

Need form links in the template

May need user rights management

Proposal Review Workflow Example1. User submits a proposal via a form

1. Fill in basic information about the proposal

2. Set a field in the form to be, say, “New”

3. User doesn’t see further information about “Reviewer” or “Owner”, etc.

2. Semantic notification (email and RSS feed) facilitates communication

3. Reviewer finds the form in a canned query1. Reviewer changes the status to, say, “Approved”

2. Reviewer field (say “Reviewed by”) is (automatically) set

3. Then reviewer assigns the proposal to a owner, say, “Alice”

4. The owner (“Alice”) now sees it and can start work on it1. Changes the status to “Active”

2. Adds a start date or maybe an estimated end date too

Ways to Handle Workflow

Add a link to open another formForce edit “review” using another form (e.g. reviewer form)

When saving the new form, the category is changed

Now we have a different category a different default form

Change a value to include or exclude to another template

May need #if (or #switch) statement in template to change (or include/exclude) template values

Use “Page has default form” property (Semantic Forms extension)

More WorkflowGood summary by Yaron

SMWCon Spring 2011 TalkSMW and the Workflow Puzzle

My Semantic Page Creator Talk on Friday 9/23

Example: A simple HelpDesk wiki

Case 3: Wiking Dev Wiki

Our project management wiki

We use it for *all* things it can do:Proposals, ideas, email messages, features, bugs

Agile development: milestones, sprints, stories, tasks

Code/Feature association: SVN mapped to tasks/bugs

Progress analysis: burn-down charts, work calendar

Collaboration, Documentation, Demonstration

Examples of Data I/O

Wiking Development Wiki

An Imported emailhttp://wiking.vulcan.com/dev/index.php/Demo_scenarios

An email imported as a Project Bughttp://wiking.vulcan.com/dev/index.php/Issue_11886_Insert_a_single_property

A task uploaded via Outlook with multiple commits

http://wiking.vulcan.com/dev/index.php/WikiTags_Release_Structure

Microsoft Office Connector

Leverage Microsoft Office applications and technology

Bring SMW info to Office applications on-demand

API for data I/O: add and modify wiki data within Microsoft Office

Utilize semantics to improve relevance

Smart actions for semantic properties

Connections

API

API

SmartJesse Wang | SemTech 2010

Our Agile Project Management (Scrum)

59

Sprint Backlog

Sprint Output

Sprint2-3 week

BurnupBurndownVelocity

Daily ScrumEvery 24 hours

Bac

klog

Mee

ting

mem

o

Etc

...

Code commit

Spr

int o

utp

ut p

ack

ageCreate stories and tasks

Mail upload

Task /

bug

upda

te

Generate daily report

User specified notificiations

Product documentation

Wiking

Jesse Wang | SMWCon Amsterdam 2010

Wiki Data I/OWiking development wiki has two external applications

Subversion integration

Microsoft Office integration

Extensions to support itSemantic Wikitags

Semantic Connector

Data I/O Extensions

MediaWiki API – very basic read/write

Page Object ModelFirst Data I/O extension to allow access some wiki structure data (links, title, basic template field)

SMWWriterBased on POM, support annotations (semantic properties)

Data API extension in SMW+

Semantic WikiTags and Semantic Connector

Wiki Object ModelA powerful and comprehensive object model (Data API)

Read/write at finer object level (sentences, template parameters)

62

Case Study: Battle-space

Luminary System Discover when New Information represents a change in understanding of entities

Discovery of explicit entity links, implicit relationships

Large Volumes of Data in various formatsUnstructured news articles

Tactical Reports, Field Intelligence

Structured Database Information

Use Wiki Pages to represent current knowledge about an entity – “what we know”

Domain Ontology to represent domain of information – “what we want to know”

Issue Alerts when Significant Events occurNew information according to category

Changing information on topics of interest

Need to send information to various devices – cell phones, email, etc.

63

System DesignWiki Configuration

Semantic MediaWiki: Large developer community, active development, open source. Wikipedia uses MediaWiki, so scalability and performance are important.

Semantic Results Format: Provides various rich media displays of semantic information, including graphs, timelines, maps

Semantic Forms: Provides convenient user interface for entering semantic data into wiki, avoiding cumbersome wikitext

Semantic Notifications: Enables sending of notifications when results of semantic query change.

Domain OntologyCreated OWL Ontology for Terrorism

Semantic Parsing, Extraction, ReasoningJava Process using various Open-Source Toolkits

Rapid plugin of new technologies

Multiple Data Sources supported

64

Sample Content Page

65

Wiki Content Design

Use Templates to Ensure Consistent Look-and-FeelTemplates Correspond to Ontology Classes

Fields within Templates correspond to Properties within Ontology

Rich Content Visualizations derived in consistent way

Hierarchical Categories match Class Hierarchy within Ontology

Ensures Validity for Properties

Category included on each Template page to ensure consistency

Forms Provide ability for users to enter data directly into wiki without knowing Wiki Text

Each form corresponds to a Template

Fields within forms correspond to the fields/properties within the Template

GUI can include auto-completion

Created Page immediately linked semantically to rest of Wiki

66

Sample Visualizations

Visualizations automatically

created w/o user edit

(tables, timelines, maps, social networks…)

UI enables notifications based on results of query – message sent when visualization changes

Optimize

Usability

Performance

Search Marketing

UsabilityMediaWiki Usability

Semantic MediaWiki UsabilityFact Box

ExtensionsSemantic Forms

One Click Way

Auto-complete

SRF

PerformanceCaching or no caching

Squid

Global Load Balancing Service

Batch ProcessRefreshData.php

Triple Store performance

Search Engine Optimization

General SEO rules (Official Google Guide 32pp)

MediaWiki Robot.txt : edit, talk, websvn, …

Metadata: title, description, media, alt tags…

Semantic MediaWikiRDFa/Rich Snippets/Search Monkey

For more info:Wil Smith: Search Engine Optimization for SMW

SummaryJumpstart

Deployment of your bundle/suite/package

Code, pseudo-code/meta-data, and data

CustomizeSkins, Styles, Extensions

Data, Code, Code, Data

Integrate Know the requirements and data

Put everything together and evolve

Acknowledgement

Jesse Wang | SMWCon Amsterdam 2010

Paul Allen

Mark Greaves

Project Halo

Daniel Hansch

Denny & Markus @ K.I.T.

Ontoprise GmbH

William Smith

Ed Swing @ VSTI (SAS)

TeamMersion LLC

72

Thank youQuestions and/or comments?

Slide uploaded at http://www.slideshare.net/jiaxinwang/

Email me at jessew @ vulcan.com or twitter @aiwang