![Page 1: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/1.jpg)
Convergence of multimedia andknowledge technologies
aceMedia, Aim@Shape, BOEMIE, MESH,X-Media, K-Space, VITALAS and VICTORY
Practitioner Day CIVR 2007
Yiannis KompatsiarisMultimedia Knowledge Laboratory
CERTH - Informatics and Telematics Institute
![Page 2: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/2.jpg)
22
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Outline• Introduction• Content - applications• A common view
• Multimedia Ontologies• Analysis• Reasoning• Retrieval
• Common issues• Dissemination• Conclusions
![Page 3: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/3.jpg)
33
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
DIRECTOR
SCENE
TAKE
TITLE
Multimedia Content
Networks
Storage & Devices
SegmentationKA Analysis
Labeling
Cross-mediaanalysis
Context
Reasoning
MetadataGeneration &
Representation
Content adaptation anddistribution - MultipleTerminal & Networks
Hybrid / Content-basedretrieval recommendations
and personalization
Semantic technologyin MarketsWeb 2.0 photo -
video applications
![Page 4: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/4.jpg)
44
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Need for annotation + metadata
“The value of information depends onhow easily it can be found,
retrieved, accessed, filtered ormanaged in an active, personalized
way”
![Page 5: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/5.jpg)
55
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Content KnowledgeExtraction Applications
3D
Industrial
Personal Sports
Semantic Desktop Retrieval
News
CommercialPersonalization
Mobile
Fashion News
![Page 6: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/6.jpg)
66
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Content KnowledgeExtraction Applications
Personal
PersonalizationMobile
Retrieval
Commercial
![Page 7: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/7.jpg)
77
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Content distributionand adaptation
Sharing
ACE concept
Actionable content
![Page 8: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/8.jpg)
88
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Content KnowledgeExtraction Applications
Retrieval
News
PersonalizationMobile
![Page 9: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/9.jpg)
99
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
News Syndication
Multi-National &Local news providers
![Page 10: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/10.jpg)
1010
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Content KnowledgeExtraction Applications
Industrial
Semantic Desktop Retrieval
![Page 11: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/11.jpg)
1111
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Large-Scale content
Process support
Industry content
![Page 12: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/12.jpg)
1212
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Content KnowledgeExtraction Applications
3D
Retrieval
![Page 13: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/13.jpg)
1313
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - ApplicationsMaximise
automation ofthe shape
knowledgelifecycle
raw data
geometric model
structural model
semantic model
conceptual sketch
shape facets andsemantic mapping
semanticallystructured model
From raw data to semantics From semantics to model
Embe
ddin
g se
man
tic c
onte
nt
Extr
actin
g se
man
tic c
onte
nt
geometric model
![Page 14: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/14.jpg)
1414
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Content KnowledgeExtraction Applications
SportsRetrieval
![Page 15: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/15.jpg)
1515
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Automatic semanticannotation of digital
mapsEVOLVED
ONTOLOGY
INITIALONTOLOGY
POPULATION &ENRICHMENT COORDINATION
INTERMEDIATEONTOLOGY
ONTOLOGY EVOLUTION
EVENTSDATABASE
MAPSDATABASE
MAP ANNOTATIONINTERFACE
SEMANTICSEXTRACTION
RESULTS
OTHERONTOLOGIES
SEMANTICS EXTRACTION
MULTIMEDIACONTENT
FROM VISUALCONTENT
FROM NON-VISUALCONTENT
FROM FUSEDCONTENT
ContentCollection(crawlers,spiders, etc.)
![Page 16: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/16.jpg)
1616
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Content KnowledgeExtraction Applications
Personal SportsRetrieval
News
![Page 17: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/17.jpg)
1717
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
K-Space
R&D
Dissemination
Fellowships
Integration
Network ofExcellence
Emphasis onintegration of
research activities
![Page 18: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/18.jpg)
1818
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Content KnowledgeExtraction Applications
Retrieval
News
Personalization
Fashion NewsLarge-Scale
Real use cases
![Page 19: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/19.jpg)
1919
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Content - Applications
Content KnowledgeExtraction Applications
3D
Retrieval
Mobile
P2P and Mobile
![Page 20: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/20.jpg)
2020
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
ManualAnnotation
- Models
AdditionalAnalysis
Information
SingleModalityAnalysis
SemanticAnalysis
Knowledge Infrastructure(Multimedia Ontology)
Knowledge ExtractionA common view
Implicit KnowledgeSignal Level
Explicit Knowledge – Logic - Semantics& Hybrid Level
![Page 21: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/21.jpg)
2121
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
AdditionalAnalysis
Information
Knowledge Infrastructure(Multimedia Ontology)
ManualAnnotation
- Models
SemanticAnalysis
SingleModalityAnalysis
Knowledge ExtractionA common view
Feature extractionText, Image analysisSegmentation, SVMsEvidence generation“Vehicle”, “Building”
Classifiers fusionGlobal vs. LocalModalities fusion
Context “Ambulance”
ReasoningFusion of annotationsConsistency checking
Higher-levelconcepts/events
“Emergency scene”
Multimedia contentannotation tools
Training(Statistical)Modeling
Domain Multimedia content
AnnotationsAlgorithms - Features
Context
AdditionalAnalysis
Information
Knowledge Infrastructure(Multimedia Ontology)
ManualAnnotation -
Models
SemanticAnalysis
Single ModalityAnalysis
![Page 22: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/22.jpg)
2222
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Use of ontologies
• Metadata representation• Annotation• Interoperability
• Reasoning• Extracting higher-level
annotations• Consistency checking• Fusion
• Ontology-driven analysis• Retrieval• Personalization
![Page 23: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/23.jpg)
2323
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Multimedia Ontologies
• Multimedia content structure• aceMedia(MPEG-7, RDF), AIM@SHAPE(3D content)
• Multimodality• MESH(OWL), BOEMIE (OWL-DL)• K-Space, X-Media (COMM, OWL, DOLCE)
• Fuzziness• K-Space, X-Media (Fuzzy-OWL)
• Changing knowledge• BOEMIE (evolution)• X-Media (versioning, reasons of change)
• Specific domains
![Page 24: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/24.jpg)
2424
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
COMM: Core Ontology of MultiMediaK-Space, X-Media
• Instead of translating MPEG-7 1-to-1into an ontology, COMM provides 5multimedia design patternswhich formalize basic notions ofmultimedia annotation
• Digital data pattern• Decomposition pattern• Content annotation pattern• Media annotation pattern• Semantic annotation pattern
• Usage of DOLCE as modeling basisand consideration of commonrequirements for multimediaannotation
• COMM already covers large parts ofMPEG-7 (some additional patternsmay be required for completecoverage)
DigitalData
MultimediaData
OutputSegmentRole
plays
ProcessingRole
InputRoleOutputRole
plays
SegmentDecompositionAlgorithm
SegmentationAlgorithm setting
satisfiesSituationMethod
InputSegmentRole
DOLCEdefines
Decomposition pattern
DigitalData
MultimediaData
AnnotationRole
plays
ProcessingRole
InputRoleOutputRole
plays
Annotation
Method
setting
setting
satisfiesSituation
defines
Algorithm
AnnotatedDataRole
DOLCE
Description
StructuredDataDescription
MPEG 7Descriptor
Content annotation pattern
![Page 25: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/25.jpg)
2525
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Multimedia Information ObjectsMESH
hasDecomposition
about
orderedBy
interpretedBy
realizedBy
„Segmentation-Tool“
MultimediaFile
Format: JPG, UTF-8Lang: DE
Format
Linguistic-IO / Visual-IO
MatchTeam
Decomposition
hasSegment
text, image
Text
hasContent
TextSegmentspatio-temporal-region
hasSegment
Image
hasContent
1 2
1 2
about
instanceOf
Example: Analysis of a Multimedia Web Document
![Page 26: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/26.jpg)
2626
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Multimedia Content AnnotationM-Ontomat-Annotizer (aceMedia, K-Space)
![Page 27: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/27.jpg)
2727
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Multimedia Content Analysis• MPEG-7 widely used for LL features• Segmentation and feature extraction tools (aceMedia, K-Space)• Well-known classifiers applied and developed
• SVMs, EM, HMM• Bio-inspired approaches
• Increasing use of context (aceMedia, K-Space)• Spatial, Frequency, EXIF
• Fusion• Classifiers (K-Space, MESH: global+local)• Modalities
• X-Media (Text+Image+1D data)• MESH (Text+Speech+Video)• aceMedia (Text+Video)
• Mostly statistical and machine learning (implicit) based but also• Hybrid (implicit + explicit, K-Space)
![Page 28: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/28.jpg)
2828
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Context and Reasoning for AnalysisaceMedia
KAA
beach scene
person
faceperson/facedetection
sceneclassification
<RDF /> rocksky
sea
beach beach/rock
rock/beach
sea, sky
person/bear
…other analysis methods
Creation ofcontextualinformation
multimediareasoning
•Use of contextual information•From metadata layer•spatio-temporal relations•Domain knowledge
•Reduction of label sets•Merging of segments
![Page 29: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/29.jpg)
2929
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Natural-Person: 0.456798Sailing-Boat: 0.463645Sand: 0.476777Building: 0.415358Pavement: 0.454740Road: 0.503242Body-Of-Water: 0.489957Cliff: 0.472907Cloud: 0.757926Mountain: 0.512597Sea: 0.455338Sky: 0.658825Stone: 0.471733Waterfall: 0.500000Wave: 0.476669Dried-Plant: 0.494825Dried-Plant-Snowed: 0.476524Foliage: 0.497562Grass: 0.491781Tree: 0.447355Trunk: 0.493255Snow: 0.467218Sunset: 0.503164Car: 0.456347Ground: 0.454769Lamp-Post: 0.499387Statue: 0.501076
Classification ResultsaceMedia
Segment’shypothesisset
![Page 30: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/30.jpg)
3030
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Semantic Region MergingK-Space
RSST
SemanticRSST
Sky
Building
Sea
Ground
![Page 31: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/31.jpg)
3131
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Cross Media Knowledge AcquisitionX-Media
Cross Media approaches:• Result level: combining results
obtained from different systemson different types of media
• Extractor level: using systemresults from different types ofmedia as annotation orbackground knowledge
• Feature level: using featurescoming from different media.
Cross Media Framework:• Multimedia Document
Processing:• Extract single & cross media features
• Feature Processing:• Find optimal representation of
feature space• Cross media data models
• Create knowledge models for allconcepts
• Cross media dependency models• Integrate background knowledge &
exploit causality information
![Page 32: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/32.jpg)
32Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Reasoning• Support of imprecision - uncertainty• Logic-based approaches
• Extensions of formal theories (X-Media, K-Space)• Ad-hoc solutions based on crisp reasoners (aceMedia)
• Statistical approaches (X-Media)• Used for:
• Fusion• Consistency checking• Higher-level results
![Page 33: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/33.jpg)
3333
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
region1region3
region2
image1
Remarks• Annotations (fuzzy ABox) considered:
• fuzzy, positive, concept assertions• crisp role assertions
• Prior knowledge (TBox) considered:• Crisp inclusion and equivalence axioms
1. Classical (crisp) DL reasoning applied on assertions, leaving out fuzzy degrees
2. Axioms revisited by external module to updateappropriately the degrees according to fuzzy interpretation semantics:
Ad-hoc Fuzzy ReasoningaceMedia
![Page 34: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/34.jpg)
34Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Fuzzy OWLK-Space, X-Media
• Automatically Derived Multimedia Annotation is oftenimprecise or errorprone• Model this uncertainty
• Extend OWL with fuzzy A-Box• “region4 shows an object which is
a ball with a fuzzy degree of 0.8 anda pumpkin with a fuzzy degree of 0.3”
![Page 35: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/35.jpg)
3535
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Σ
Abduction as non-standard retrieval: Acquire what should be added toa knowledge base (Σ,Δ) to make a query (set of assertions) Γ true
Interpretation as abduction: let Γ be the analysis produced concept and roleassertions, Γ1 : by default assertions, Γ2 : ones that need to be explained
TBox ABox
Σ includes apart from DL axioms, rules to answer the Γ queries in a backward-chaining way.
Multiple explanations may be obtained.•Consistency checking eliminates not valid answers•Preference measure according to the number of new assertions required
Γ2
Δ1: adds 2 new individuals and infers Pole_Vault
Δ2: adds 1 new individual and infers Pole_Vault
Δ3: adds 1 new individual and infers High_Jump(neglects though pole1:Pole)
Possible explanations
ΒΟΕΜΙΕ
![Page 36: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/36.jpg)
3636
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
RetrievalVITALAS, Victory
VITALAS:• Adapting the search space to the user profile and providing
interactive functionalities to control the results• The system validation will be performed on professional
collections, up to 10,000 hours of video (television archives –INA/IRT) and 1.500.000 still images (Belga)
• The textual annotation would have different interpretationregarding the usage context.
VICTORY:• 3D and multimedia distributed content (MultiPedia) into Peer-to-
Peer (P2P) and mobile Ρ2Ρ networks• 3D content (and context) analysis, personalization, 3D object
watermarking techniques
![Page 37: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/37.jpg)
3737
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
aceMedia applications
Web
-based
Stan
dalo
ne
aceMedia PCapplications
![Page 38: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/38.jpg)
3838
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Common (Open) Issues
• Evaluation• Annotated content• Ontologies• Fusion in analysis• Uncertainty in reasoning• Large-Scale• Generic vs. Specific approaches• Multiple domains support
![Page 39: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/39.jpg)
3939
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Dissemination Activities
• SMART: Semantic MultimediA Research andTechnology, networking cluster
• SAMT: International Conference on Semanticsand digital Media Technologies (EWIMT)• 2007: 5-7 December 2007, Genova, Italy
• SSMS: Summer School on MultimediaSemantics• 2007: Glasgow, UK, July 15-21, 2007
• Special issues, sessions, workshops, books
![Page 40: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/40.jpg)
4040
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Conclusions
• Semantic analysis of multimedia isalready providing results
• Fundamental and applied research in• Logic-based + signal approaches• Implicit + explicit (knowledge) approaches
• Different applications and requirements• Ongoing research in all areas• Future direction: analysis+reasoning for
social (Web 2.0) applications
![Page 41: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/41.jpg)
4141
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Many thanks to theprojects!
![Page 42: Convergence of multimedia and knowledge technologies · Metadata Generation & Representation Content adaptation and distribution - Multiple Terminal & Networks Hybrid / Content-based](https://reader034.vdocuments.site/reader034/viewer/2022042805/5f5dd6581fb56571396227bd/html5/thumbnails/42.jpg)
4242
Multimedia Knowledge LaboratoryInformatics and Telematics Institute
Thank you!CERTH-ITI / Multimedia Knowledge Laboratory
http://mklab.iti.gr