[email protected] chosun univ. the study on the semantic image retrieval using the cognitive...
TRANSCRIPT
CHOSUN UNIV.
The Study on the Semantic Image Retrieval Using the Cognitive Spatial Relationships in the Semantic Web
Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim
CHOSUN UNIV.
Contents
• Introduction• Related Works• Our Approach• Test and Experimental Results• Evaluations• Conclusion
CHOSUN UNIV.
Introduction
• Huge number of data in the web• Image data is rapidly increasing• Object Based Spatial Relationships
VS Cognitive Spatial Relationships• Building a Spatial Relationships
Ontology
CHOSUN UNIV.
Related Works
• Information Retrieval System– Keyword Matching – Very important technique on the web
environment– Process the various information items
• Text Documents, Images, Sounds and etc.
– Generally, accuracy is low
CHOSUN UNIV.
Related Works
• Ontology based Image Retrieval– Try to solve the heterogeneous between
the terminologies– Need the extra works
• Creating and Maintaining the ontologies
– It is still unsuitable for the image retrieval system• Because it doesn’t consider the features of
the images
CHOSUN UNIV.
Related Works
• The Spatial Description Logic– Region Connection Calculus : RCC-8– Spatial representation is regular subsets of the
topological space– Elementary binary relationships between the
regions• PO, NTPP, TPP, EQ, TPP-1, NTPP-1, EC, DC
X Y X Y YX XY
X Y X Y Y XXY
DC(X, Y) EC(X, Y) TPP(X, Y) TPP- 1(X ,Y)
PO(X, Y) EQ(X, Y) NTPP(X, Y) NTPP- 1(X, Y)
CHOSUN UNIV.
Our Approach
• Background Knowledge of the Cognitive Spatial Relationships
CHOSUN UNIV.
Our Approach
• Building Process of the Spatial Relationships Ontology– Defining the Cognitive Spatial
Relationships– User Research– Using WordNet and Dictionary– OWL Representation
CHOSUN UNIV.
A B
A B
A B
BA
BA
AB
AB
A(B)
C(A,B)
DC(A,B)
PO(A,B)
TPP(A,B)
NTPP(A,B)
TPP- 1(A,B)
NTPP- 1(A,B)
EQ(A,B)
A B
A B
A B
C(A,B)
DC(A,B)
PO(A,B)
RCC- 8
Cognitive Spatial Relationships defined
in our study
Our Approach
• Definition of the Cognitive Spatial Relationships
CHOSUN UNIV.
Our Approach
• User Research– 200 images– 10 people– Clustering the spatial words
CHOSUN UNIV.
Our Approach
• Architecture of the spatial ontology– Upper level– Basic spatial
words level– Instance level
Cognitives_r
connect partof disconnect
verb
v_c v_p v_d
proposition
p_c p_p p_d
kiss lie run swim ride look jump fall
Bussosculatexxxx
Restxxxx
Speed
hurryzipxxxx
Gomovetravelxxxx
Sitdrivexxxx
Frontfaceseexxxx
Leapboundspringxxxx
Pursuenearxxxx
onacrossthroughalongxxxx
atin
aroundroundxxxx
overunderbesidenearxxxx
Upper_Level Cognitive Spatial Relationships
Spatial VerbsBased on the Research
Spatial Verbs Based on the WordNetSpatial Propositions Based on
the OXFORD Dictionary
CHOSUN UNIV.
Our Approach
• WordNet and Dictionary Matching
Cognitive spatial relationships
Research words WordNet matching words
connect Attach Connect, link, tie, link up, fasten, touch, adjoin, meet, contact
connect Kiss Buss, osculate
disconnect Chase Chase after, trail, tail, tag, give chase, god, go after, pursue, follow
disconnect Jump Leap, bound, spring
partof Float Drift, be adrift, blow, swim, transport
partof Hide Conceal, shroud, enshroud, cover, obscure, blot out, obliterate, veil
The spatial propositions based on OXFORD Dictionary
connect On, along, across, through
disconnect Over, under, above, below, by, beside, near, before, behind
partof At, in, around, round
CHOSUN UNIV.
Test and Experimental Results
• System Architecture– Contents provider interface– Ontology part– User interface
CHOSUN UNIV.
Test and Experimental Results
• Test Environment– Queries
1. Only one word query – e.g. swan2. Two words query – e.g. swan and lake3. Query containing the spatial relationships –
e.g. swan in the lake4. Natural Language query containing the
spatial verbs – e.g. swimming swan5. Natural Languages query containing the
spatial verbs and proposition – e.g. swan swims in the lake
CHOSUN UNIV.
Test and Experimental Results
• Accuracy Measurement
• Experimental Results
Accuracy = All images searched throughout the system
Correct images matched with the query
CHOSUN UNIV.
Conclusion and Future Works
• Definition of the Cognitive Spatial Relationships
• Applying them to the Image Retrieval System
• Still have Limitation : Semi-Automatic• Our study presents the vision of the
semantic image retrieval and natural language query processing
