elis – multimedia lab multimedia lab rik van de walle department of electronics and information...

ELIS – Multimedia Lab

Multimedia LabRik Van de Walle

Department of Electronics and Information SystemsGhent University - IBBT

Gaston Crommenlaan 201 bus 8B-9050 Ledeberg-Ghent, Belgium

t: +32 9 33 14914 f: +32 9 33 14896

m: +32 478 39 62 31 t secr: +32 9 33 14911

e: [email protected]: multimedialab.elis.ugent.be

MMLab OverviewF2F MMSEM, Athens – December 2006 2/28


Multimedia Lab - UGent

• History– background in image and signal processing– 1999: Multimedia Systems and Applications

new research area within Ghent University

– 2001: Multimedia Lab formally foundednew research group within ELIS department

– 2003: link/co-operation with IMEC– 2004: partner of the IBBT

www.ibbt.be, Flemish Government




• People– 5 staff members

• Rik Van de Walle, full professor• Peter De Neve, 10% lecturer• Erik Mannens, project management• Peter Lambert, senior researcher• Ellen Lammens, administrative management

– currently 25 researchers• PhD students• “others” (projects)

– about 15 master students per year




• Teaching– programs

• ba/ma Computer Science (Faculty of Engineering)• ba/ma Electronics (Faculty of Engineering)• ba/ma Informatics (Faculty of Sciences)

– Multimedia Techniques (baC3 - baE3)– Development of MM Applications (maC1 - maE1)– Advanced MM Applications - project (maC2 -

maE2)– Multimedia (baINFO2)– Internet Technology (baINFO3)



Research topics: data + metadata

place: Sydneybuilding: opera housearchitect: Jørn Utzontype: still imageformat: JPEG2000

opera: il barbiere di Sivigliacomposer: Gioachino Rossinidirector: Claudio Abbadotype: audioformat: MP3

opera: il barbiere di Sivigliacomposer: Gioachino Rossinilibretto: Cesare Sterbinitype: web pageformat: html



Research topics: adaptation - optimization

networknetwork

multimediamultimedia terminalsterminals

INTERACTION DEPENDING ONmono/stereo audio, color depth, resolution, bandwidth, ...



Research topics (1)

• Advanced video applications– development and exploitation

of scalable video compression techniques– motion estimation and compensation– ROI-based video compression (Region of

Interest)– adaptation of multimedia data

with respect to varying usage environments– development of iDTV applications

(a/o based on MHP, the Multimedia Home Platform)



Research topics (2)

• Mobile multimedia applications– reduction impact of network losses

on quality of multimedia presentations(a/o adaptive coding/decodingwrt varying network characteristics)

– rendering multimedia dataon mobile terminals with limited resources

– transparent handover of multimedia sessions between different devices (session mobility)

– hardware/software co-designof embedded multimedia systems



Research topics (3)

• Standardization– MPEG-21

• Digital Item Declaration• Digital Item Adaptation• Digital Item Processing• Development of Reference Software for various tools

– MPEG-4 Scalable Video Coding (SVC)– MPEG-4 Advanced Video Coding (H.264/AVC)– JVT - Joint Video Team– VQEG - Video Quality Experts Group– IFTA - International Federation of Television

Archives



Ex. 1: exploitation of scalability


ELIS – Multimedia LabEx. 2: session mobility for mobile

applicationsstudent watchinga lesson on a set-

top box

student continuingthe session on a

PDA

finishing the e-learning session

on a tablet PC

tv/set-topbox

PocketPC

Session Transfer

Session Transfer

Media stream

Media stream

Media stream

Tablet PC

Serverscontaining

onlineclasses


ELIS – Multimedia LabEx. 3: video adaptation for mobile

applications



In short: transparent content adaptation



New Research Pathsrelated to MMSEM



Feature Extraction (Video)

• Use cases: – Video surveillance

• Detection of moving foreground objects

– Video adaptation• Change bitrate, framerate, resolution, coding format,

…– goal: reduction of network load, processing load,

adjustment to client capacities or user preferences

• Extraction of relevant features: ROI, semantics, shot detection



Feature Extraction tool

• Low level extraction:



Integrated Video Surveillance System

H.264/AVC

Digital Item

Item

Item

Component

Descriptor

Item

Descriptor

Component

Component

DescriptorDescriptor

Raw Pixel Data

MPEG-21 Digital Item

features features

• Combining expertise:



Interoperable Integrated Video Surveillance System

• Issues– Which features (high level <-> low level,

automatic <-> manual)– How to describe them– When to describe them (camera, video

processing module, network node, client)– How to link them to video resource (storage,

distribution, presentation, …)

New Use Case?



New IBBT Projectsrelated to MMSEM



PeCMan

• The goal of the PeCMan project is to research the major technological challenges of a user-centric solution enabling uniform storage and manipulation of data as well as universal access to this personal data. A successful solution will have premium capabilities in the areas of security, performance, usability and description of the personal data


Manual annotation Automatic annotation

+ detailed fast

- time consuming, mostly not uniform

error prone, semantic gap

Overcome semantic gap?

• Extract the ”good” features -> which algorithm?, domain specific?• Combine which primitive features for scene understanding?• Machine learning -> many methods, training on different datasets, ...• Ontologies -> need to model the whole world? Content vs. Context

Personal Digital Photo Archives

--> need for effective, uniform annotation strategy

semantic gap is still very wide!



Feature extraction- human vision system best tuned to color and texture perception- imitate cells of the visual cortex- texture features can provide semantic information - texture helpful in object segmentation



Machine learning

--> use both manual and automatic annotation with minimal user overhead

Machine Learning- supervised <-> unsupervised - self-organizing map: processing principles of human (visual & auditory) cortex- incremental learning: knowledge must be continuously updated to manage non-static archives, relevance feedback

Feature extraction

Scene understandingScene understanding

OntologiesClassification



Lindo



FIPA / AVEQ

• investigate the processes related to engineering and analysis of audiovisual material, based on the notion of a virtual model of the product

• explore some aspects of reverse engineering, including audiovisual-quantization, analysis and classification

• Stretching the idea of the virtual model, we may develop a much more powerful analysis and classification system when we succeed in reconstructing the model – the background, the characters, the objects that constitute the scene of the image – quantifying the subjects per scene by a finite list of normalized and relevant vectors and being able to express semantics, history and behaviour in a single geospatial and temporal reference



Industrial partners

• Main partners (projects & MPEG)– Alcatel, France-Belgium– Artec/Televic, Belgium– Barco, Belgium– Belgacom, Belgium– CISCO/Scientific Atlanta, USA– Concentra, Belgium– Deutsche Telecom, Germany– eTampere, Finland– France Telecom, France– IBM, USA– Intel, USA– LG Electronics, Korea



Industrial partners

• Main partners (projects & MPEG)– Microsoft Research, USA– Museon, Netherlands– Philips, Netherlands– Porthus, Belgium– ScreenPeaks, Israel– Siemens, Germany– Sun Microsystems, USA– Telindus, Belgium– Thomson Multimedia, France– T-Systems Nova GmbH, Germany– VMMa, Belgium– VRT, Belgium



Contact

Erik Mannens

Department of Electronics and Information SystemsGhent University - IBBTGaston Crommenlaan 201 bus 8B-9050 Ledeberg-Ghent, Belgium

t: +32 9 33 14992 f: +32 9 33 14896 m: +32 478 39 62 31 t secr: +32 9 33 14911

e: [email protected]: multimedialab.elis.ugent.be

elis – multimedia lab multimedia lab rik van de walle department of electronics and information...

Documents