MultimediaMultimediaMuch More than just a Buzzword!Much More than just a Buzzword!

Alan HanjalicAlan Hanjalic

Coordinator, Theme MultimediaCoordinator, Theme Multimedia

ICT Delft Research CenterICT Delft Research Center

Defining MultimediaDefining Multimedia

Where does the “buzzwordness” come from?Where does the “buzzwordness” come from?

Multimedia is the ability to (dis)play sound, images,… !!?!Multimedia is the ability to (dis)play sound, images,… !!?!Multimedia is gadgets !!?!Multimedia is gadgets !!?! … …??

The use of computers to present text, graphics, video, animation, and sound in an integrated way. Long touted as the future revolution in computing, multimedia applications were, until the mid-90s, uncommon due to the expensive hardware required. With increases in performance and decreases in price, however, multimedia is now commonplace. Nearly all PCs are capable of displaying video, though the resolution available depends on the power of the computer's video adapter and CPU.

Definition of the term “Multimedia” from www.webopedia.com :

Power of MultimediaPower of Multimedia

Multimedia versus Multiple MediaMultimedia versus Multiple Media Emphasis onEmphasis on Synergy Synergy of single media!of single media!

Enabling Paradigm Shifts in many fieldsEnabling Paradigm Shifts in many fields PC-like HCI PC-like HCI Natural HCI and smart environmentsNatural HCI and smart environments Classroom Education Classroom Education Remote and personalized educationRemote and personalized education Written meeting notes Written meeting notes Multimedia meeting summaryMultimedia meeting summary Staring at 1000 monitors Staring at 1000 monitors Smart autonomous surveillanceSmart autonomous surveillance TV Zap TV Zap Personalized TV Anytime AnyplacePersonalized TV Anytime Anyplace

Now is THE time!Now is THE time! Mature and affordable digital capturing, processing, storage and Mature and affordable digital capturing, processing, storage and

communication technology and standardscommunication technology and standards

Natural HCINatural HCI

Natural Natural Human like Human like Human-like communication Human-like communication Multimedia communication Multimedia communication SynergySynergy

of what we seeof what we see of what we hearof what we hear of what we touchof what we touch of what we knowof what we know of how we feelof how we feel of how we interpret thingsof how we interpret things

Challenges in natural HCIChallenges in natural HCI High level of “synergetic” interactivityHigh level of “synergetic” interactivity

Transparently using all modalities!Transparently using all modalities!

High level of anticipationHigh level of anticipation How to learn the synergy?How to learn the synergy?

Natural HCINatural HCIProactive, optimally structured interfaces

“Pick up all the ammunition in the room, drink something, and then take the door on your left”

Experience Multimedia Natural HCI

Smart EnvironmentsSmart Environments

Ambient intelligence: A definitionAmbient intelligence: A definition * * a seamless environment that is constantly aware of the a seamless environment that is constantly aware of the

presence of people, their needs and desires, and allows natural presence of people, their needs and desires, and allows natural interaction via intuitive and natural user interfaces such as interaction via intuitive and natural user interfaces such as gestures, speech, etc. gestures, speech, etc.

Ambient Intelligence, as a convergence ofAmbient Intelligence, as a convergence of Ubiquitous/pervasive computingUbiquitous/pervasive computing Ubiquitous communicationUbiquitous communication Intuitive and intelligent Intuitive and intelligent

user interfaces user interfaces

Characteristics of AmICharacteristics of AmI Embedded, aware, natural, Embedded, aware, natural,

personalized, adaptive, anticipatory personalized, adaptive, anticipatory

*after M. Rauterberg, TU/e

It’s all about the Context Awareness!

www.ics.forth.gr

Smart EnvironmentsSmart Environments

Basic enabling Basic enabling technology mature technology mature and affordable:and affordable: Sensors Gadgets Displays Embeddable CPUs Communication

bandwidth

Smart Environments:Smart Environments:We can do more!We can do more!

Suspiciousness model:Synergy of modalities

SensorAudio

Video

Level of suspiciousness

time

Threshold

Potential event of interest

Alert! Alert! Alert! Alert!

Threshold setting based on desired alertness level

Low-level prior knowledge

Relevance feedback

Context awareness Multimedia Smart environment

Smart camera

Autonomous & Personalized Surveillance and Monitoring

Entertaining and Informing PeopleEntertaining and Informing People

Goal:Goal: Enabling easy, non-linear access to content, independent of Enabling easy, non-linear access to content, independent of

content amountcontent amount Help people deal with content overloadHelp people deal with content overload Inform people as effectively as possibleInform people as effectively as possible

Digital Video Broadcasting Digital Video Broadcasting “New Style” * “New Style” *

Huge amount of video hours instantaneously accessible by the user

Broadcasters transfer to digital video Broadcasters transfer to digital video production, transmission, and production, transmission, and receiving chainreceiving chain

Growing popularity and affordability of Growing popularity and affordability of high-capacity digital storage devices high-capacity digital storage devices at consumers at consumers Expected penetration of up to 70% within Expected penetration of up to 70% within

10 years in the EU market10 years in the EU market

* PVR, Home Media Servers and Future of Broadcasting, Durlacher Research LTD.

TV Broadcast ConsumptionTV Broadcast Consumption“New Style” *“New Style” *

Death of the TV ChannelDeath of the TV Channel Different understanding of the broadcasting mechanismDifferent understanding of the broadcasting mechanism The concept of a channel will lose its meaningThe concept of a channel will lose its meaning

From “Zap TV” to “Local” On-Demand & Personalized TVFrom “Zap TV” to “Local” On-Demand & Personalized TV Broadcasted material will be recorded routinely and automaticallyBroadcasted material will be recorded routinely and automatically Programs will be accessed from the local storagePrograms will be accessed from the local storage Viewing of life TV is likely to diminish with the timeViewing of life TV is likely to diminish with the time

New problem to solveNew problem to solve How to deal with enormous number of content hours available to How to deal with enormous number of content hours available to

consumers?consumers?

* PVR, Home Media Servers and Future of Broadcasting, Durlacher Research LTD.

Content Indexing and RetrievalContent Indexing and RetrievalNews report on Microsoft Weather report

Video segmentation and classification/indexing

Commercial break

Affectiveuser profile

Mood profile of a movie

Affectcurve

Moviematches

userprofile

Recommendation

Personalizedcontentdelivery

Content Multimedia Easy Access

An extension of the same Concept:An extension of the same Concept:Content Access on Mobile DevicesContent Access on Mobile Devices Holy grail:Holy grail:

Getting the content we like, wherever and whenever we like!Getting the content we like, wherever and whenever we like!

Why not? Enabling technology is there!Why not? Enabling technology is there! Broadband mobile communications Broadband mobile communications Mobile devices with high-resolution video displayMobile devices with high-resolution video display Mobile CPUs of sufficient powerMobile CPUs of sufficient power Powerful batteriesPowerful batteries

Exciting new applicationsExciting new applications Personalized news anytime & anywherePersonalized news anytime & anywhere Personalized sport highlights anytime & anywherePersonalized sport highlights anytime & anywhere

Marketing possibilityMarketing possibility Offering a service to a broadcaster for analyzing, filtering, Offering a service to a broadcaster for analyzing, filtering,

adapting and delivering content to mobile clientsadapting and delivering content to mobile clients

Multimedia R&DMultimedia R&DWhat does it take to succeed?What does it take to succeed?

Knowledge of theoretical fundamentalsVision on technological possibilities

Universities

Knowledge of marketVision on future applications

Resources for application development

IndustrySynergy

Contribution from ResearchContribution from Research Booming research in the Multimedia field in the recent yearsBooming research in the Multimedia field in the recent years ACM SIGMMACM SIGMM

ACM Multimedia Special Interest Group created 10 years agoACM Multimedia Special Interest Group created 10 years ago Both universities and large industrial research labs involved (IBM, Intel, Both universities and large industrial research labs involved (IBM, Intel,

Microsoft, HP, Mitsubishi Electric, Sharp, …)Microsoft, HP, Mitsubishi Electric, Sharp, …) ACM SIGMM Strategic RetreatACM SIGMM Strategic Retreat

Organized periodically to assess the current state of multimedia Organized periodically to assess the current state of multimedia research and suggest directions for future researchresearch and suggest directions for future research

Three grand challenges suggested by the retreat:Three grand challenges suggested by the retreat: ““Make interactions with remote people and environments nearly the Make interactions with remote people and environments nearly the

same as interactions with local people and environments”same as interactions with local people and environments” ““Make authoring complex multimedia titles as easy as using a word Make authoring complex multimedia titles as easy as using a word

processor or drawing program”processor or drawing program” ““Make capturing, storing, finding and using multimedia an everyday Make capturing, storing, finding and using multimedia an everyday

occurrence in our computing environment” occurrence in our computing environment” Dutch universities are among the world leaders in this field! Dutch universities are among the world leaders in this field!

Searching for Industrial PartnersSearching for Industrial Partners

Industrial partners needed for valorization that areIndustrial partners needed for valorization that are Proactive and flexible – inherent in a good SME!Proactive and flexible – inherent in a good SME!

Recognize new opportunities on time and go for them Recognize new opportunities on time and go for them

Ready to get involved in early stagesReady to get involved in early stages Go for joint ventures (BVs, spin-offs) with universitiesGo for joint ventures (BVs, spin-offs) with universities

Open minded and visionaryOpen minded and visionary Ready to accept longer development trajectoriesReady to accept longer development trajectories

Benefit from the “metamorphosis” of Dutch universities:Benefit from the “metamorphosis” of Dutch universities: No “Ivory Tower” any moreNo “Ivory Tower” any more Eager to see use of its research results in new products and Eager to see use of its research results in new products and

servicesservices Eager to create value from research and introduce paradigm shifts Eager to create value from research and introduce paradigm shifts

in the world of consumers and professionalsin the world of consumers and professionals

What Could be a Success Story: What Could be a Success Story: WKGemist.nlWKGemist.nl

What Could be a Success Story: What Could be a Success Story: FabplayerFabplayer

“Powered by TU Delft”

To conclude …To conclude …

MultimediaMultimedia By no means a Buzzword, but rather an Eye Opener!By no means a Buzzword, but rather an Eye Opener!

US & Asia recognized potential of Multimedia long time agoUS & Asia recognized potential of Multimedia long time ago Difficult to catch up with on the international market!Difficult to catch up with on the international market!

But, lots of possibilities for domestic market!But, lots of possibilities for domestic market! Huge variety of ideas for Multimedia Products and Services can Huge variety of ideas for Multimedia Products and Services can

successfully be valorized successfully be valorized

Highest potential for valorizationHighest potential for valorization Media production and authoringMedia production and authoring Media consumption (at home or at mobile devices)Media consumption (at home or at mobile devices) Smart private and work environmentsSmart private and work environments

So, what are we waiting for?So, what are we waiting for?