exposing semantics to drive transcoding

Exposing Semantics To

Drive Transcoding

Darren Lunn, Sean Bechhofer and

Simon Harper

2/28

Summary

• Visual Rendering Can Provide Semantic Information

• Semantics Can Be Used to Drive a Transformation Process

• Poor Design Decisions Can Hinder Flexibility and Adoption

• A Well Defined Model and a Little Pragmatism Goes a Long Way

3/28

The Web

• Focuses on Presenting Information in a

Visual Manner

– Images

– Columns

– Chunks

• Some Knowledge is only Available

Implicitly from the Page Rendering

4/28

Implicit Knowledge

Advertisement

Banner

Menu

Main Content

5/28

Assistive Technology

• Visually Impaired Users use Assistive

technologies e.g. Screen Readers

– Render Pages Sequentially in Audio

– Achieved by Accessing the Underlying HTML

• Focus on Visual Presentation Rather than

Content Hampers This

– Particularly if Attention is Not Paid to Coherent Design

– Subtleties of Visual Presentation Can be Lost

6/28

Accessing CNN In Audio

7/28

Screen Readers

• Traversal of Content is Serial– Top-to-bottom

– Left-to-Right

• Important Information may not be Encountered Until Later On.

• Information Such as Menus may be Repeated for Every Page– Tiresome if the User has to Wait for the Menu for Each

Page

8/28

SADIe Approach

Banner

Banner

Menu

Headline Story Overview

Tabs

Advertisement

Image

Main Story

Heading

Story Overview

Main Story

9/28

Original Annotation Solution

• Use an Ontology as an Abstraction to

represent Basic Concepts Appearing in

the Page

• Annotate the CSS Rather than the Page

10/28

Proposed Solution

HTML

CSS

OntologyTransformed

Page

Rendered

Page

11/28

Two-Part Ontology

• An Upper Ontology Provides Basic

Information about Authoring Concepts

• This is Extended to Provide information

about Particular Style Sheets

• The Definitions in these Ontologies

Provide the Annotation of the CSS

Elements

12/28

Architecture

Application

HTMLCSS

Upper Ontology

Site-Specific Extension

13/28

Overarching Aim

• Describe Semantic Structure of

Websites

• Use Inference Engines to Determine

Relationships Between Elements of the

Website

• Transcode Website Based on these

Relationships

14/28

Success?

• Transcoding was Successful on a

Diverse Range of CSS-based Sites

• User Studies Demonstrated the

Usefulness of the Transformations

• But the Model had Weakness that

Limited The Approach

15/28

Application Functionality vs.

Semantic Structure

SADIe

Removable NonRemovable

Priority Menu

Medium LowHigh

16/28

Application Functionality vs.

Semantic Structure

Menu

Removable

High Priority

17/28

Separate Functionality From

Domain Knowledge

• We Advocate Separation of Structure

(HTML) from Presentation (CSS)

• Also Separate Knowledge from

Application Functionality

– Split Ontology Into Two

– Push More Computation into the Application

18/28

Why Separate

• Adds Flexibility to the Overall

Application

• Adds Flexibility to the Overall Approach

• Easier For Designers to Construct

19/28

General Relationships

• CNN_AdBox is Contained Within a

2ColumnFloat

• 2ColumnFloat is Removable Therefore

CNN_AdBox is Removable

• What If CNN_AdBox is Not Removable?

<pclass=“2ColumnFloat”>

<divclass=“CNN_AdBox”>...</div>

</p>

20/28

Class Containment

High Priority Removable

CNN_AdBox 2ColumnFloat

isContained Within

hasPriority isRemovable

21/28

Only Within That Instance

• CNN_AdBox is Contained Within a

2ColumnFloat Only Within this Instance

• CSS Properties Can Still be Applied

Even If CNN_AdBox is Not Contained

Within 2ColumnFloat

22/28

A Little Testing Goes A Long

Way

• Small Scale Testing Brings to Light

Errors of Modelling

• Can not Expect To Know Everything

Without Real World Case Studies

• Prevents Effort of Reengineering

• Prevents Loss of Faith in the Tool

23/28

Be Pragmatic

• Designers are Not Ontology Engineers

• Significant Overhead Will Hinder

Adoption

• Balance Between Minimal Effort but

Enough Knowledge to not be Hindered

24/28

New SADIe Architecture

Application

HTMLCSS

Structural Ontology

25/28

CSS Role Property

• Add a New Role to the CSS

• Still Validates

• Explicitly States What the Class Represents

• Values Based on WAfA Ontology– Ontology of Accessibility and Web Authoring Concepts

2ColumnFloat{

-uom-structural-role:LinkedMenu;

...

}

26/28

Benefits

• Less Overhead to Expose Semantics

• Non Destructive

• Still Provide Same Functionality

– New Transformations being Investigated

• Other Uses of the Exposed Semantics

– AiSC

27/28

Conclusion

• Visual Rendering Can Provide Semantic Information

• Semantics Can Be Used to Drive a Transformation Process

• Poor Design Decisions Can Hinder Flexibility and Adoption

• A Well Defined Model and a Little Pragmatism Goes a Long Way

Questions

http://hcw.cs.manchester.ac.uk