Download - THEORETICAL ISSUES IN CONTEXT MODELLING
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 1
THEORETICAL ISSUES IN THEORETICAL ISSUES IN CONTEXT MODELLINGCONTEXT MODELLING
Kalervo Järvelin([email protected])
University of TampereFinland
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 2
Talk OutlineTalk Outline
Lab IR - traditional study designs and contributions
Contexts of IR Modeling context - issues in modeling A cognitive approach to modeling Consequent research designs Conclusion - does it matter?
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 3
The Lab IR FrameworkThe Lab IR Framework
Docu-ments
Docu-ments
Represen-tation
Represen-tation
DatabaseDatabase
Searchrequest
Searchrequest
QueryQuery
MatchingMatching
Represen-tation
Represen-tation
QueryResult
QueryResult
EvaluationResult
EvaluationResultEvaluationEvaluation
Relevance assessment
Relevance assessment
Recallbase
Recallbase
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 4
The most popular focus -it is CS, after all, isn’t it?!
IR Research Goals IR Research Goals
Traditional goals: (a) theoretically (~formally) understanding
IR, (b) empirically testing IR methods, and (c) designing better IR systems
This can be seen in typical IR study designs - by populating the framework with variables
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 5
Lab IR Study Types / DesignsLab IR Study Types / Designs
Study types: experimental methodological constructive theoretical
Experimental designs – IR evaluation focus areas with variables designs operate with
these fringe areas controlled
(no analytical variables) dependent variables: ER many hidden variables
DocumentsDocuments
Rep MethodRiRep MethodRiRep SetDiRep SetDi
TopicTopic
Query VersQjQuery VersQjMatching MethMkMatching MethMk
Rep MethodTjRep MethodTj
EvaluationProcedurePeEvaluationProcedurePe
Relevance AssessmentRelevance Assessment
RecallBaseRecallBase EvaluationResult EREvaluationResult ER
QueryResult AijkQueryResult Aijk
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 6
LabIR: Hypos, Laws, Theories LabIR: Hypos, Laws, Theories
The dependent variables - typically recall and precision - using various metrics
The independent variables typically are the use or non-use of various techniques
The controlled variables - collection, requests
Thus Lab IR is about the explanation of the variation of recall and precision through IR techniques applied
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 7
LabIR: ContributionsLabIR: Contributions IR technology Theoretical growth
theory expansion, e.g., due to enrichment of indexing theory
greater analytical power through formal model building improved empirical support through testing over several
test collections new research questions – within the given set of
variables
Theoretical growth has been limited by not bringing in radically new concepts or relations
Cannot possibly analyze real world applicability
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 8
Lab IR Tests Lab IR Tests
IR research typically considers only retrieval tasks which most often are: (a) purely topical (b) content-only (c) well-defined (d) static, and (e) exhaustive
This is like saying that no matter what your situationis, your needs always arepurely topical, content-only, well-defined, static, and …… True, isn’t it?
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 9
Small World?Small World?
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 10
Psssst ...... IR in Context
Alternatives?
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 11
One possible context:Task PerformanceAugmentation through IR
Augmenting Task Performance
Colleagues
Knowledge Methodology
Education/Training
Create Information Seeking
Remember/Use
Tools
Work Task Situation
Other SourcesEducation
Documents
Information Systems
Acquire Knowledge
Find DocsDocs Available
Retrieve Docs
Other Collections
Colleagues ...
DocDB FactDB
Context
Web
...
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 12
Socio-organizational& cultural context
Nested Context FrameworksNested Context Frameworks
How much should one cover?
Work Task
WorkProcess
Task Result
Work task context
Seeking Task
SeekingProcess
Seeking Result
Seeking context
Docs
Repr
DB
Request
Query
Match
Repr
ResultIR context
How to model what is covered?
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 13
What to cover?What to cover?
Clearly, current IR test designs are not representative of real world IR
In principle, whatever has important relationships with the
application of IR, should be covered however, what has, is not well known answering this requires theoretical modeling,
operationalization and empirical testing so, what is context?
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 14
Context? What?!Context? What?! Dervin (1997):
there is no term that is more often used, less often defined, and when defined defined so variously as context - it has become almost a ritualistic invocation
for some, context has the potential of being virtually anything that is not defined as the focus
for others, it is inextricable surround that denies all generalizations
there are endless lists of contextual factors This is why we hate it - it is foreign to CS!
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 15
Some Contextual ThemesSome Contextual Themes
Reality is discontinuous, knowledge partial Context is not useful as an independent
entity Context requires a focus on process and
relationships between product and process Need to focus on multiple interdependencies Context is a necessary source of meaning
(Dervin 1997)
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 16
How to model? - IR Context How to model? - IR Context DimensionsDimensions1. Work task dimension2. Search task dimension3. Actor dimension4. Perceived work task dimension5. Perceived search task6. Document dimension7. Algorithmic search engine dimension8. Algorithmic interface dimension9. Access and interaction dimension
Each dimensioncontaining multiple
variables
Much more detailin the forthcomingbook by Peter&Kal
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 17
http://www.springeronline.com/
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 18
InformationObjects
InformationSystems
InterfacesCognitiveActor(s)
Organisat.
Cultural
= Cognitive transformation and influence
= Interactive communication of cognitive structures
Social
&4
3 2 1
5
6
8
7
Context
A Cognitive Model of IS&RA Cognitive Model of IS&R
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 19
Context around and withinContext around and within
Informationobjects
IT: EnginesAlgorithms
InterfaceInformation
seeker
Org.
Cultural
SocialContext
Real Social & PhysicalWorld
Cognitive World
PerceivedInformation
objects
PerceivedIT: EnginesAlgorithms
PerceivedInterface
Situationi.e. perceivedtask – need
sources
Social
Org. / Cultural
Perceived context
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 20
Augmented IR Research Goals Augmented IR Research Goals
Theoretically understanding the application of IR in / to context; not just engines but also their use in varying situations
Empirically describing / testing the application of IR in / to various contexts
Supporting the design of information systems, services and information management
This does not downgrade IR into Social Science but rather means more / better engineering
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 21
New Research SettingsNew Research Settings
Extended laboratory experiments What kinds of work tasks are typical in Domain X? What kinds of search tasks do these generate? What kinds of documents tend to be relevant?
IIR experiments Lab approach but including the seeking actor
Research on interfaces / actors in context interaction of actors and information system
interfaces in diverse socio-organizational contexts
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 22
Lab IR settings focused on Lab IR settings focused on domain/task domain/task variables variables and dimensions of and dimensions of ITIT and and information objectsinformation objects
Informationobjects
IT
TaskOrg.Context
Interface Actor(s)
Social
Cultural
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 23
Lab IR settings focusing on Lab IR settings focusing on actor actor variables and variables and dimensions of dimensions of ITIT and and information objectsinformation objects
Informationobjects
IT
TaskOrg.Context
Interface
Social
Cultural
Work taskperception
Search taskperception
Actortypes
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 24
ConclusionConclusion Context is endless and pervasive
its management requires perspective, goals and interests: what do we want to achieve??
we may see multiple, nested contexts Thus, what is IR about? As technology ...
developing systemsbased on a technological imperative?based on people’s communication / access needs?
As a discipline ... [vs. a technology] understanding ... perhaps ... but what? C-n-c-
l?
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 25
Does context really matter?Does context really matter?
We do not know really (if, or how much) Some certainly wish / believe it does not I don’t believe them until shown that
no matter what one’s situation is, one’s information needs are of just one type
no matter of what type information needs are, they are always best served in one already known way
Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 26