project I2RPIntelligent Information Retrieval and Presentation
in public historical multimedia databases
prof. dr. L. Schomaker
KI/RuG
2
ToKeN2000 grants for research between computer science, AI
and cognitive science
money from Min. of Econ. affairs and Min. of Education
demonstrating that the ‘human perspective’ has an added value
demonstrating that working systems and/or models can be implemented
3
ToKeN2000Project Title
EIDETIC Intelligent Content-based Image Retrieval
I2RP Intelligent IR and Presentation in public historical multimedia databases
DUMPERS Distributed User Modeling and Exploration in Personalized Recommender Systems
CHIME Cultural Heritage in an Interactive Multimedia Environment
AUTHENTIC Knowledge discovery and disclosure for visual art: authentication and dating of graphic art and paintings
ANITA Administrative Normative Information Transaction Agents
VINDIT Combining visual and textual information for IR
MIA Medical Information Agent
DIME Distributed Interactive Medical Exploratory for 3D Medical Images
TIMEBAYES Building and Using Temporal Bayesian Models in a CPR setting
NARRATOR Narrative disclosure of health-care knowledge
4
I2RP partners
CWI
Universiteit Leiden
Universiteit Maastricht
Rijksuniversiteit Groningen
Rijksmuseum Amsterdam
5
prof. L. Hardman CWI/TUE
prof. dr. H.J. van den Herik UM
prof. dr. G.A.M. Kempen UL
prof. dr. L.R.B. Schomaker RUG
dr. I. Sprinkhuizen-Kuyper UM
dr. J. van Ossenbruggen CWI
dr. N. Taatgen RUG
Supervisors
+ Rijksmuseum: dhr. K. Schoemaker
6
Researchers
drs. Stefano Bocconi OIO CWI
dr. Floris Wiesman postdoc IKAT
drs. J. Grob OIO RUG
drs. C. van Breugel UL
+ M.Sc. students
7
Intelligent Information Retrieval and Presentation
Information Retrieval: searching in weaklyorganized multimedial databases
Presentation: user and context-relatedrendering of retrieved results
“Intelligent”, i.e., making use of methodsfrom AI and Cognitive Science
Upper-left picture is the query
“boy in yellow raincoat”
…yields very counter-intuitive results
What was the user’s intention?
9
Human-machine communication
Grice’s Maxims of bi-directional cooperative dialog: quantity (adapt the size of your answer) quality (tell the useful truth) relation (react to what has been asked) manner (avoid ambiguities)
Current HMC violates most of these maxims
10
Starting points in I2RP Bidirectional cooperative dialog (Grice) (maxims of quantity, quality, relation, manner)
An example of ‘intelligent information retrieval and presentation’: car sales
Buyer: “I’m looking for a Volvo 850 Estate for less than 5000 Euro”
11
Starting points in I2RP Bidirectional cooperative dialog (Grice) (maxims of quantity, quality, relation, manner)
An example of ‘intelligent information retrieval and presentation’: car sales
Buyer: “I’m looking for a Volvo 850 Estate for less than 5000 Euro”Seller: “we don’t have it” (logical response)
12
Starting points in I2RP Bidirectional cooperative dialog (Grice) (maxims of quantity, quality, relation, manner)
An example of ‘intelligent information retrieval and presentation’: car sales
Buyer: “I’m looking for a Volvo 850 Estate for less than 5000 Euro”Seller: “we don’t have it” (logical response)
vsSeller: “we do have a Mitsubishi Station of 5500 Euro” (intelligent response)
13
Reasoning with world knowledge
(1) Volvo 850 Estate (3) Mitsubishi Station
(2) family car!
all cars
sports cars SUVs
14
Knowledge sources in I2RP
A bi-directional cooperative dialog (Grice)…
Requires: world knowledge semantic web, ontologies knowledge on humans user modeling, language
15
Project Partners
Optima: A user agent for object-based image search
Spreekbuis: A Dutch sentence generator
Cuypers: Automatic user-centric hypermedia generation
GO: Graphical Ontologies
16
Spreekbuis: a sentence generator for Dutch
UL (C. van Breugel/Arsenijevic)
Performance Grammar Workbench (PGW)
17
Optima: a user agent for object-based image search
KI/RuG, Taatgen/Grob/Schomaker
User modeling , learning in ACT-R
KI
RuG
18
Cuypers: user-centered hypermedia generator
CWI
Stefano Bocconi, AIO per 01-01-2002
using knowledge on graphical design and communication in the application domain
19
GO: Graphical Ontologies
IKAT/UM (Floris Wiesman)
‘Generic tool for searching (navigating), accessing, and editing ontologies’
MetaBrowser: a graphical browser for information retrieval
20
Goal of the meeting
a lot of mono-disciplinary research exists
… based on toy problems or artificial data
(TREC, multimedia retrieval benchmark dBs)
… barely looking at the user requirements
I2RP we can do it better!
21
System: application/experimentation
RenderingSemantics
User Modeling Speech/Language
Multimedia retrievalapplication
22
System: application/experimentation
Multimedia retrievalapplication
dB
UI
Optima/ACT-R
GO Cuypers
Spreekbuis
23
Dependencies
RenderingSemantics
User Modeling Speech/Language
dB
UI
Optima/ACT-R
GO Cuypers
Spreekbuis
24
Agenda
Group introduction
Bilateral discussions
Integration
Concrete goals: define Milestones Experimentation-platform specification Demonstrable output
25
Agenda bilateral 20-min. discussions
Room C001 UM + RuG UL + RuG UM + UL
Room C002 UL + CWI UM + CWI CWI + RuG