1 list – dtsi – service réalité virtuelle, cognitique et interfaces sensorielles a conceptual...
TRANSCRIPT
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles 1
A Conceptual Approach to A Conceptual Approach to Web Image RetrievalWeb Image Retrieval
Adrian PopescuAdrian PopescuGregory GrefenstetteGregory Grefenstette
CEA LIST, FranceCEA LIST, France
OntoImage 2008OntoImage 2008Marrakech, MoroccoMarrakech, Morocco
Contacts : [email protected]
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles 2
Some facts
60% of Internet image queries include only one conceptPeople are stupid or lazy?
What are they looking for?Common nounsPlacesOther (famous) peopleProductsSexual content
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
3May 2008DTSI
Our Image Browsing Strategy
Introduce conceptual hierarchy into image search applications
Expand common words by more specific terms
Present results reflecting hierarchy Dual browsing: image or text query Exploit conceptual hierarchy and image
spidering to structure image base
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
4May 2008DTSI
Architecture of OLIVE
Olivepicture retrieval
framework employing a large scale conceptual hierarchy
to provide a dual access to Web images
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
5May 2008DTSI
Adapting WordNet
Concept rankingBased on Web counts
Relevance (C1) = webCount(C1, parent)
Web counts and WordNet structure Relevance (C1, C2) = webCount(C1, parent) *
dist(C1,C2)/senses(C1)
Example for dog (Web counts) pooch, pug, Newfoundland, basset (Web+WordNet) collie, basset, german shepherd, doberman
Query reformulationQuery concept + representative leaf
nodes
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
6May 2008DTSI
Knowledge for dog
Organism, canine, domestic animal, living thing, physical entity
Parents
Wolf, fox, hyena, wild dog, domestic cat, bitch, jackal
Related types
Poodle, corgi, spitz, cur, hunting dog, working dog, toy dog, Dalmatian, griffon
Narrower concepts
pooch, pug, Newfoundland, basset, beagle, cairn, Airedale, Doberman, German shepherd
Leaf Subtypes
yesPolysemic
Cat, monkey, elephant, tiger, lion, golden retriever, kittens, Chihuahua, snake
Expanded search
Rabbit, bird, Scooby DooRelated names
Puppy, Alaskan huskyNarrower
search
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
7May 2008DTSI
First response page for dog - Google Image
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
8May 2008DTSI
First response page for dog - Olive
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
9May 2008DTSI
First response page for Doberman - Olive
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
10May 2008DTSI
Visual similarity search for image of Doberman
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
11May 2008DTSI
Evaluation – Precision on first two pages
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
12May 2008DTSI
Precision over ten pages
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
13May 2008DTSI
Qualitative evaluation
Experiment setting 10 users compared Olive and Google
Image Questionnaire about: results structuring;
enriched interaction; automatic reformulation;
Positive reactions from the users to the new features introduced in Olive
The users also provided useful comments under the form of free text.
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
14May 2008DTSI
Conclusions
A semantic retrieval architecture can cover a significant part of Web picture queries.
Dual access to picture content, textual and visual
Simple but efficient solution to introduce CBIR in large scale architectures
With semantic structures, existing image indexes can be better used than in current applications
OLIVE outperforms Google Image, qualitatively and quantitatively
Future work: The extension of the hierarchy (e.g. using
Wikipedia)
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
15May 2008DTSI
A glimpse of the future
LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles
16May 2008DTSI
Advertising
Websitehttp://moromete.net
I am looking for workStarting November 2008