Types and Methods of Content Adaptation

T-110.456 Next Generation Cellular Networks

Timo-Pekka Viljamaa

2.3.2005

Agenda

• Motivation for Content Adaptation

• Types of Content Adaptation

• Methods of Content Adaptation

• Real Life Examples

• Conclusions

• Further Reading

Motivation for Content Adaptation

• Terminal diversity• Different display size, bandwidth, memory, processing power, UI,

etc.

• Available network environment• Depending on the current location and the terminal support

• UMTS/GPRS/GSM Data

• User preferences• Information presenting styles

• e.g. scrollable/splitted to several views• Time issues

• e.g. see utilitarian content quickly/wait for flashy content• Cost issues

• Definition of Multimedia Unit (MMU)• The unit of data transmitted over a network containing one or

more multimedia content elements• e.g. multimedia message in MMS, elements of a web page

(typical composed of several MMUs)

Types of Content Adaption

• Format Adaption

• Characteristics Adaptation

• Appearance Adaptation

• Size Adaptation

• Encapsulation Adaptation

Format Adaption

• Converting original content format to the format also understandable by the receiver

• e.g. JPG to GIF, MPEG4 to MPEG, AAC to MP3

football.jpg football.gif

Characteristics Adaptation

• Modifying media object’s characteristics while remaining within a given format

• e.g. image/video resolution, frame/bit rate, number of colors

590 x 758

295 x 379

Appearance Adaptation

• Modifying the content of an multimedia unit (MMU) for the purpose of changing how it looks or sounds

• Needed to conform to the capabilities of the receiver and may even increase usability

• e.g. condensed version from a web page, portrait to landscape mode

Google (Opera for Mobile S60)

Google (Internet Explorer)

Size Adaptation

• Reducing the size of an MMU message to match the capabilities of the receiver and the underlying network environment:1. Removal of some MMU objects

• e.g. remove an image from an MMU message

2. Changing the encapsulation• e.g. split an MMU message to several smaller ones

3. Converting to another format• e.g. MP3 to AAC

4. Characteristics adaptation• e.g. quality of the image, number of colors

Encapsulation Adaptation

• Converting MMU messages from one ”application protocol” to another and therefore involving repacking a message without altering any of the media content• e.g. splitting an e-mail to a sequence of several SMS

messagesFrom: sender@foo.com

To: receiver@bar.com

foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar

E-mail

foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar

From: sender@foo.com

To: receiver@bar.com

foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar

SMS x 2

Methods of Content Adaptation

• Multimedia Transcoding

• Content Selection

• Rendering at the Client

• Hybrid Approaches

Multimedia Transcoding

• Modifying the properties of multimedia object to meet the capabilities of the terminal

• Usually automatic process• The behavior of the adaptation system is programmed in

advance• Can include all previously presented types of adaptation

• Multimedia Transcoding Architecture

Capability negotiationExtracts capabilities info from protocol, e.g. from UserAgent Header or UAProf

Capability DBContains the capabilities of different terminals.

Adaptation Policies Eng.Decides how the content should be adapted.

Media Adaptation Eng.Applies transformation to the media content or selects the right version.

Policies Plugins Media Plugins

GIF JPEG

WML H.263

MMS Browsing

SIP IM Rich calls

Application-specific Controller

Requests/Responses

Multimedia Transcoding (2)

• Transcoding of audio visual content• Decode object Modify uncompressed Encode to desired

format• Also partially decoded objects can be modified • Scalable formats improve the quality and the performance

• Transcoding of nonaudiovisual content• Nonaudiovisual content is often XML applications

• Can be modified and and/or converted to other XML applications using XSLT technology or DOM/SAX interfaces

• Transcoding of procedural code• Complexity makes transcoding unattractive• Better to make the code itself adaptive or provide different

versions

• Advantages and drawbacks+ Increased usability+ Automatic process- May require a lot of processing resources- Adapted results may not be acceptable or usable- Copyright issues

Content Selection

• Multiple versions or modality of each multimedia object is stored in the server and server selects the best version for the given terminal

• The Infopyramid• A representation scheme providing a multimodal,

multiresolution representation hierarchy for multimedia content

• Modality axis provides the same information under different media modes

• Resolution axis provides, for a given modality, the content at different quality levels

Text Image Video Audio

Full story

Short story

Title160x120

40 kbps

8 kpbs

QVGA

VGA

96 kbps

1 Mbps

64 kbps

128 kbps

Modality

Res

olut

ion

Content Selection (2)

• The Customizer• After a client’s request selects the best content representation

supported by the client terminal’s capabilities and environment from the Infopyramid

• Makes content selection decisions in accordance with adaptation policies

• Seperating content and its representation• Same content with several representation alternatives• e.g. CSS and XSLT

• Advantages and drawbacks+ Selecting less processing intensive than transcoding+ Increased usability+ Automatic process after the creation of all versions+ The quality of delivered content has been validated by the

author+ Solves legal issues- May require a lot of work from the author to manage the content- May require a lot of space to store all the different versions

Rendering at the Client

• Adapting the content before the user sees it on the screen by rendering it in a suitable way for the given terminal

• Theorically can fit content to variety of different screen size• Binary content can be adapted by the renderer

• The renderer also handles alternative content• e.g. alternative text instead of an image if ”Show images”

turned off

• Advantages and drawbacks+ The client knows its capabilities best and has the

up2date information+ Content is received on the terminal and can be

readapted if needed- Requires processing resources in the terminal Not suitable

for low-end devices

Hybrid Approaches

• Mixing different adaptations methods together at the same time• e.g. transcoding can be a part of a content selection system,

transcoding and content selection can be performed on the media content and the final layout is left to the terminal when rendering

Real Life Examples

• Apache Cocoon• http://cocoon.apache.org/

• EU Project Consensus• http://consensus-online.org/

• Nokia MMS Solution• http://www.nokia.com/nokia/0,,56882,00.html

• Opera for Mobile, Small-Screen Rendering™ technology• http://www.opera.com/products/mobile/smallscreen/

Conclusions

• Multimedia transcoding works well for automatic adaptation of simple media content

• But often fails when the content is more sophisticated and requires more processing resources

However only option if the client doesn’t perform adaptation• Content selection gives the author more control on the adapted

versions of the content• But requires knowledge of target terminal and some work to

create the different versions and establish the selection rules Who will do these versions and keep them up2date?

• Rendering at the client should work well because the client obviously knows its capabilities best and has the up2date information about itself

• But requires large amount of processing resources in the terminal

Is the consuming of processing resources worth it and what to do with low-level devices?

• No best method for adapting content that suits all situations

A hybrid approach might work best

Further Reading

• Adapting Multimedia Internet Content for Universal Access (Mohan, R., Smith, J.R., Chung-Sheng Li)

• http://ieeexplore.ieee.org/xpl/abs_free.jsp?arNumber=00748175

• MPEG-21 Multimedia Framework• ISO/IEC TR 2100-1:2004

• http://www.iso.org/iso/en/CatalogueDetailPage.CatalogueDetail?CSNUMBER=40611&ICS1=35&ICS2=40&ICS3=

• MPEG-21: Goals and Achievements (Burnett, I., Van de Walle, R., Hill, K., Bormans, J., Pereira, F.)

• http://ieeexplore.ieee.org/xpl/abs_free.jsp?arNumber=1237551• Interoperable Adaptive Multimedia Communication (Timmerer C.,

Hellwagner, H.)• http://ieeexplore.ieee.org/xpl/abs_free.jsp?arNumber=01377105

• SVG Mobile (SVG Basic & Tiny)• http://www.w3c.org/TR/SVGMobile/

• Flash Lite• http://www.macromedia.com/software/flashlite/

Questions?

Thank you!