types and methods of content adaptation - aalto · 9.11.2005 t-110.5120 next generation wireless...

Types and Methods of Content Adaptation

Anna-Kaisa Pietiläinen

9.11.2005T-110.5120 Next Generation Wireless

Networks 2

Agenda

IntroductionMultimedia Content TypesTypes of AdaptationMethods of AdaptationConclusion

9.11.2005T-110.5120 Next Generation Wireless

Networks 3

Introduction

9.11.2005T-110.5120 Next Generation Wireless

Networks 4

The ProblemTerminal diversity

Hardware: display size, resolution, bit rate, CPU power, storage space, user interface, colour depth, ...Software: supported encodings, formats and protocols

Network characteristics and coverageBluetooth, GPRS, WLAN, UMTS, ...

User preferencesWhich tradeoffs to make?

Research going on under the name pervasive or ubiquitous computing

9.11.2005T-110.5120 Next Generation Wireless

Networks 5

Multimedia Content Types

Media content Presentation contentApplication dataProcedural code

9.11.2005T-110.5120 Next Generation Wireless

Networks 6

Media ContentTextual content

encoding (ASCII, UTF-8, UTF-16)Audiovisual content

speech, audio, music, images, video, graphicusually binary encodedNonscalable content ie. the encoded format provides only one representation of the content. Examples: GIF, JPEG, PNG, WBMP, MPEG-4 AAC, MP3, ASF, AMR (+ other speech codecs), MPEG-4, H.263 and so on...Scalable content ie. contains many representations in a single bit stream. Examples: JPEG2000, SVG (SVG Basic and Tiny for mobile terminals), etc.

9.11.2005T-110.5120 Next Generation Wireless

Networks 7

Presentation ContentMarkup languages

HTML, WML, XHTML, XHTML Mobile Profile, SMILHTML in theory device independent, offers possibilities to define alternate contentXHTML 2.0 has advanced alternate content object functionalitySMIL is a media description language and allows also to define alternatives and to control when to use them

Stylesheetsseparation of data and its representationCascading Style Sheets (CSS)Extensible Stylesheet Language Transformations (XSLT)

Emerging technologiesXHTML 2.0, XForms, CSS Media Queries

Working groups at World Wide Web ConsortiumHTML, Mobile Web Initiative, Device Independence

9.11.2005T-110.5120 Next Generation Wireless

Networks 8

Application Data

Application specific contentcalendar entries (vCalendar)contact information (vCard)spreadsheetssynchronization data (SyncML)

Usually XML formattedPortability between applications in different devices

9.11.2005T-110.5120 Next Generation Wireless

Networks 9

Procedural Code

Software as contentDevice-independent software

J2ME (MIDP + CLDC)Standard software platforms

Symbian, Palm OS, PocketPC

9.11.2005T-110.5120 Next Generation Wireless

Networks 10

Types of Adaptation

Format adaptationCharacteristics adaptationAppearance adaptationSize adaptationEncapsulation adaptation

9.11.2005T-110.5120 Next Generation Wireless

Networks 11

Format Adaptation

MotivationNew formats are emerging all the timeTerminal support varies and depends on

can terminal be upgraded terminal’s restrictionslack of business opportunitylicencing costs

Usage reduce the content sizeprovide the content in the most suitable formatprovide the content in the supported format

Has often a big impact on the content quality

9.11.2005T-110.5120 Next Generation Wireless

Networks 12

Characteristics Adaptation

Adaptation within a single media formatimage/video resolutionfile sizesampling ratenumber of frames in GIF animationframe ratenumber of colours

For example, in MMS messaging image resolution is adapted based on the receiving terminal’s capabilities

9.11.2005T-110.5120 Next Generation Wireless

Networks 13

Appearance Adaptation

Requires knowledge of object’s semantic type ie. what can be discardedExamples

reducing the amount of information on the web pagesplitting content to multiple pageslandscape versus portrait display differencies

9.11.2005T-110.5120 Next Generation Wireless

Networks 14

Encapsulation Adaptation

How the content is packaged for transmissionFor example, an email arriving to SMS gateway could be split to several SMS messages

9.11.2005T-110.5120 Next Generation Wireless

Networks 15

Size Adaptation

’Side-effect’ of the other adaptationsappearance adaptation eg. removal of some objectschange the encapsulation eg. split to several messagesconvert to another formatcharacteristics adaptation

9.11.2005T-110.5120 Next Generation Wireless

Networks 16

Methods of Adaptation

TranscodingContent selectionRendering at the clientHybrid approaches

9.11.2005T-110.5120 Next Generation Wireless

Networks 17

TranscodingModifying the properties of media objects using the various adaptationsAlso known as dynamic adaptationDone either at the originating server or at the intermediate proxyUsually an automatic process

Decoding and re-encoding of audiovisual contentXSLT or DOM/SAX based manipulations for XML formatted nonaudiovisual content

9.11.2005T-110.5120 Next Generation Wireless

Networks 18

Transcoding Methods: General Architecture

9.11.2005T-110.5120 Next Generation Wireless

Networks 19

Transcoding Methods: Content Adaptation Pipeline

Three-phase adaptation process forming a pipeline[6]

data characterization function (<- original content object)adaptation command generator (<- content characteristics and client profile, produces adaptation instructions)content adaptation executor (-> adapted object)

An implementation and performance analysis described in master’s thesis from HUT [1]Main foundings

feasibletransformations were taking most of the timeorder of transformations have an impact on the latency

9.11.2005T-110.5120 Next Generation Wireless

Networks 20

Transcoding : pros and consAutomated processDynamicStorage space benefits

ButRequires a lot of processing resourcesProcessing latencyQuality issues

HTML converted to WML might not be usableInformation loss

Copyright issues

9.11.2005T-110.5120 Next Generation Wireless

Networks 21

Content SelectionProvide multiple versions of the same media objectAlso known as static adaptationAvoids the problems of

having a lot of processing resourcesdegraded qualitylegal issues

Author has full control over the contentSelection process can be made automatic

ButComplicates content creation and managementRequires a lot of storage spaceMight be infeasible given the amount of devices, capabilites, formats etc.

9.11.2005T-110.5120 Next Generation Wireless

Networks 22

Content Selection Technologies: InfoPyramid [3,4]

9.11.2005T-110.5120 Next Generation Wireless

Networks 23

Content Selection Methods: InfoPyramid Framework [3,4]

9.11.2005T-110.5120 Next Generation Wireless

Networks 24

Content Selection Technologies : Separating the Content and Representation

Can be seen as static content adaptationone content objectseveral representations created in advance

MethodsCSS - a stylesheet for each clientXSLT - generic XML content transformed to number of formats

9.11.2005T-110.5120 Next Generation Wireless

Networks 25

Content Selection Technologies : Separating the Content and Representation

Apache Cocoon [7]Something similar can be obtained with server-side coding

Java servlets and jspApache Struts Apache Velocity

9.11.2005T-110.5120 Next Generation Wireless

Networks 26

Rendering at the ClientApplications make decisions on how to render the content

layout decisionswhich of the alternatives to usecharacteristics adaptation

image scalingcolour depthetc.

Good because terminal capabilities are known Drawback is the requirement for processing power in the low-end terminals

9.11.2005T-110.5120 Next Generation Wireless

Networks 27

Hybrid Approaches

Content is selected and transcoded based on the terminal capabilites by the server and/or an intermediate proxy leaving still the final rendering decisions to the terminalA performance study on a hybrid architecture that pre-adapted a set of content variants and left the final generation to a dynamic algorithm executed on request [5].

9.11.2005T-110.5120 Next Generation Wireless

Networks 28

ConclusionsTerminals, standards and software develope rapidly, but still often independently of each other, which creates problems in terms of content suitabilityTo increase the usability of mobile internet services, content adaptation is requiredThere are two fundamental methods for content adaptation: static or dynamic adaptationDevice independence is also important

separation of the data and its representation

9.11.2005T-110.5120 Next Generation Wireless

Networks 29

References1. S. Hairedtin, Adaptation of Content Delivery to Wireless Devices, Master’s

Thesis, HUT / Department of Electrical and Communications Engineering, 2003.2. Z. Lei, N. Georganas, Context-based Media Adaptation in Pervasive

Computing, Proceedings of Canadian Conference on Electrical and Computer Engineering, Toronto, May 2001.

3. R. Mohan, J.R. Smith, C. Li, Multimedia Content Description In The InfoPyramid, IEEE Proceeding of ICASSP, Seattle, May 1998.

4. R. Mohan, J.R. Smith, C. Li, Scalable Multimedia Delivery for Pervasive Computing, Proceedings of ACM Multimedia ’99, Orlando, October 1999.

5. W.Y. Lum, F.C. Lau, On Balancing Between Transcoding Overhead and Spatial Consumption in Content Adaptation, Proceedings of MOBICOM ’02, Atlanta, September 2002.

6. T. Phan, G. Zorpas, An Extensible and Scalable Content Adaptation Pipeline Architecture to Support Heterogeneous Clients, Proceedings of ICDCS ’02, Washington, July 2002.

7. The Apache Cocoon Project, url: http://cocoon.apache.org/8. The Device Independence Working Group, url: http://www.w3.org/2001/di/9. The Mobile Web Initiative, url: http://www.w3.org/Mobile/