event-based fusion of distributed multimedia data sources vincent oria department of computer...
Post on 30-Dec-2015
214 Views
Preview:
TRANSCRIPT
Event-Based Fusion of Distributed Multimedia Data Sources
Vincent OriaDepartment of Computer Science
New Jersey Institute of TechnologyNewark, NJ 07102
2
Outline
Classical Data Integration Problem Multimedia Data An Architectural approach to Multimedia Data
Integration Event-Based Integration of Data Sources Conclusion
3
Classical Data Integration*
* Borrowed from M. Lenzerini
4
Classical Data Integration Issues How to construct the global schema? (Automatic) source wrapping How to discover mappings between the sources and the global
schema? Limitations in the mechanisms for accessing the sources Data extraction, cleaning and reconciliation How to process updates expressed on the global schema, and
updates expressed on the sources? The modeling problem: How to model the mappings between the
sources and the global schema? The querying problem: How to answer queries expressed on the
global schema? Query optimization
5
Multimedia Data
Multimedia data management is more than physical server designLogical data modeling is important
Multimedia data management is more than similarity search
“Show me all the images that are similar to this one [in terms of color, texture, shape].”
Querying is much more complicated Give me all the news items on Baghdad over the last 2
weeks
6
Multimedia Data … Multimedia data is heterogeneous in both format
and in access primitives and this has to be accommodated
You cannot store all the data in a single DBMS; the system has to be open
Query-based access to multimedia data is important as well as browsing and some transactional access
Some DBMS-like interface and control over multimedia data should be provided
8
Multimedia Database Processing
Multimedia Data Preprocessing System
Database Processing
MM DataPre-
processor
AdditionalInformation
<!ELEMENT ..>.....<!ATTLIST...>
Multimedia DBMS
Users
Que
ry I
nter
face
MM DataInstance
<article>.....</article>
Recognized components
MM DataInstance
MM Data
Meta-Data
9
Document Database Architecture
Document Processing System
Database Processing
DTD or XML Schema files
Sch
ema
Par
ser
<!ELEMENT ..>.....<!ATTLIST...>
DTD/ XML
Manager
TypeGenerator
Document DBMS
Users
Que
ry I
nter
faceDTD/
XML Schema
Document content
DocumentsXML or SGML
DocumentInstance
ParseTree
<article>.....</article>
<!ELEMENT ..>.....<!ATTLIST...>
DTD/ XML Schema
Types
Objects
DocumentParser
InstanceGenerator
10
Image Database Architecture
Image Processing System
Database Processing
Image Analysis and Pattern Recognition
ImageAnnotation
Image DBMS
Users
Que
ry I
nter
face
Image ContentDescription
Image
Image
SyntacticObjects
SemanticObjects
<article>.....</article>
Meta-Data
11
Video Database Architecture
VideoProcessing System
Database Processing
Video Analysis and Pattern Recognition
VideoAnnotation
Video DBMS
Users
Que
ry I
nter
face
Video
KeyFrames
Meta-Data
<article>.....</article>
Video ContentDescription
Video
12
Multimedia Data Integration: An Architectural Perspective
Simple Client-Server Integrated Server Database Server Middleware and Mediation
13
Simple Client-Server
Heavy-duty client Synchronization, user interface, QoS, …
Client has to access each server Scalability problems
client code has to be updated when new servers come on-line
Meta-data
DatabaseServer
DatabaseServer
TextServerText
Server
Text
ImageServerImageServer
Images
CMServerCM
Server
Video/Audio
Client
14
“Integrated” Server
Heavy-duty server DBMS should be able to handle multiple storage systems Real-time constraints on CM
Meta-data
Video/AudioImagesText
ImageServer
CMServer
Object StorageServer
DBMS Functions
Client
15
Database Server
Lighter client Client has to access only one server Scalability problems
server may become a bottleneck - distribute and interoperate
Meta-data
CMServerCM
Server
Video/Audio
ImageServerImageServer
ImagesText
TextServerText
Server
DatabaseServer
DatabaseServer
Client
16
Document Server
Document-centric view Multimedia objects are parts of documents
Might be suitable for, e.g., e-commerce catalogs
Video/AudioImagesText
StructuredDocument
DBMS
ImageDBMS
CMDBMS
Client
17
Interoperable System
Middleware
WrapperWrapper Wrapper Wrapper WrapperText
DBMSText
DBMSVideoDBMS
ImageDBMS
ImageDBMS
Mediator Mediator Mediator
Mediator
Mediator
Client Client
18
Event-Based Multimedia Data Integration An event aims at modeling any happening
Facts, context An event has 3 components
Time Space (location) Objects
19
Events: Temporal Dimension
Time Line and Temporal relationships
Event1
Image Video
Text
Event2
Image Video
Event3
Image Video
Image
Time Line
20
Events: Spatial Dimension
GIS (Location and Spatial Relationships)
Event1Event2
Event3
Directional and Topological relationships
21
Events: Object Dimension
Which real world objects are involved in the event? Object Recognition Classical Data Integration
22
Event: Spatio-Temporal Dimension Moving Objects and their Trajectories
Raw representation: The trajectory T of a moving object is defined as a sequence
of vectors
T=[t1, …, tn] Each ri show the successive positions of the moving object over a period of time. Movement sequence:The trajectory of a moving object is represented by a sequence of (movement direction, distance ratio) pairs. This representation is not affected by rotation, shifting or scaling.
M=[m1, …, mn-1]
Each mi is a pair of (movement direction, distance ratio).
23
Event Model
Events model interpretation context Example: KIMCOE 2006 is an event
Participants are objects Location: Hilton Garden Inn, Suffolk, Virginia Date/Time: October 24 - 27, 2006 Has sub-events like sessions or visit of Lockheed Martin's
Center For Innovation Event Properties
Discrete or continuous Local or distributed Simple or composite Descriptors: Data (classical and multimedia)
24
Event Querying
Objects: RDBM, XML
Time
Space: GIS
25
Event Querying
Objects: RDBM, XML
Time
Space: GIS
26
Event Querying
Objects: RDBM, XML
Time
Space: GIS
27
Event Operators
Temporal Operator Spatial Operators Spatio-Temporal Operator Aggregation
28
Aggregation and Concept Hierarchy Dimensions are hierarchical by nature: total
orders or partial orders Example: Location(continent country
province city) Time(yearquarter(month,week)day)
Industry Country Year
Category Region Quarter
Product City Month Week
Office Day
29
Aggregation and Concept Hierarchy: Operators
roll-up (increase the level of abstraction) drill-down (decrease the level of abstraction) slice and dice (selection and projection) pivot (re-orient the multi-dimensional view) drill-through (links to the raw data)
30
Aggregation and Concept Hierarchy: Roll-up Use of aggregation to summarize at different levels of a dimension hierarchy Ex: if we are given total sales per city we can
aggregate on the market to obtain sales per state
Dayton
Q1 Q2 Q4
Drama
Horror
Sci. Fi..
Comedy
Time (Quarters)
Market(city, state)
Q3Newark
S. OrangeN. York
Category
Roll-up on Market
Ohio
Q1 Q2 Q4
Drama
Horror
Sci. Fi..
Comedy
Time (Quarters)
Market(States,,USA) Category
Q3New Jersey
New YorkArizona
31
Aggregation and Concept Hierarchy: Drill-down Inverse of roll-up
Given a total sales by state, we can ask for more detailed presentation by drilling down on market
Dayton
Q1 Q2 Q4
Drama
Horror
Sci. Fi..
Comedy
Market(city, state)
Q3Newark
S. OrangeN. York
Category
Drill-down on Market
Ohio
Q1 Q2 Q4
Drama
Horror
Sci. Fi..
Comedy
Time (Quarters)
Market(States,,USA) Category
Q3New Jersey
New YorkArizona
32
Aggregation and Concept Hierarchy: Dice and Slice
January
Slice on January
Newark
Electronics
JanuaryDice onElectronics andNewark
33
Conclusion
Event model: A data Integration model This is a work in progress: We need to fully
define the event model We want to build on existing Technology
(RDBMS, XML, GIS,..)
top related