csa3212: topic 5 © 2005- chris staff 1 of 78 [email protected] university of malta csa3212:...
TRANSCRIPT
CSA3212: Topic 5© 2005- Chris Staff
1 of [email protected] University of Malta
CSA3212:User-Adaptive Systems
Dr. Christopher StaffDepartment of Intelligent Computer Systems
University of Malta
Topic 5: User Modelling
2 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Aims and Objectives
Background to user modellingUser model implementationsTypes of user modelUnderstanding user behaviour
4 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Aims and Objectives
User-adaptive systems in general need to represent the user in some way so that the system (interface and/or data) can be adapted to reflect the user's interests, needs and requirements
The representation of the user is called a user profile or a user model
5 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Aims and Objectives
UM has its roots in philosophy/AI, and the first implementations were in the field of natural-language dialogue systems
For adaptive systems, user model must learn (at least some of the) user requirements/preferences
User models can be simple or complex, but remember that you can only get out of them what you put in!
6 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Uses of user models
Plan recognitionAnticipating behaviour/user actionsUser interestsInformation filteringUser ability…
7 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Why a user model is required in UAS
A user model is required to adapt the information space to reflect the users preferences, needs and requirements, amongst others
The level of adaptation in adaptive hypermedia systems is summarised in the following diagram, but can also apply to UASes
8 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
9 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Classifications of User Model
Two main classifications of user modelAnalytical CognitiveEmpirical Quantitative
Reference: G. Brajnik, G. Guida and C. Tasso, “User Modelling in Intelligent
Information Retrieval” in Information Processing and Management, Vol. 23, 1987, pp. 305-320
10 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Empirical Quantitative Empirical quantitative models make no effort to
understand or reason about the user Contain surface knowledge about the user Knowledge about the user is taken into
consideration explicitly only during the design of the system and is then hardwired into the system (early expert systems)
E.g., models for novice, intermediate, expert users Fit the current user into one of the stored models
11 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Analytical Cognitive Try to simulate the cognitive user processes that
are taking place during permanent interaction with the system
These models incorporate an explicit representation of the user knowledge
The integration of a knowledge base that stores user modelling information allows for the consideration of specific traits of various users
12 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Taxonomies of User Models
Rich classifies analytical user models along three dimensions Rich, E.A. (1983): 'Users are Individuals : Individualising User Models',
in International Journal of Man-Machine Studies, Volume 18. (http://www.cs.utexas.edu/users/ear/IJMMS.pdf)
Gloor, P. (1997), Elements of Hypermedia Degisn, Part I (Structuring Information) Chapter 2 (user Modeling) Section 1 (Classifications and Taxonomy).
Section reference: http://www.ickn.org/elements/hyper/cyb13.htm Book reference: http://www.ickn.org/elements/hyper/hyper.htm
13 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
1st Dimension: Canonical vs. Individual
Canonical User ModelUser model caters for one single, typical user
Individual User ModelModel tailors its behaviour to many different
users
14 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
2nd: Explicit & Implicit
Explicit User ModelUser creates model himself/herselfE.g., selecting preferences in a Web portal
Implicit User ModelUM built automatically by observing user
behaviourMakes assumptions about the user
15 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
3rd: Long-term vs. short-term
Long-term user modelsCapture and manipulate long term user interestsCan be many and variedFrequently difficult to determine to which
interest the current interest belongsInfo changes slowly over time
16 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
3rd: Long-term vs. short-term
Short-term user modelsAttempts to build user model within single
sessionVery small amount of time availableNot necessarily well defined user need
user might not be familiar with terminology
Short-term interest can become long term interest…
17 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling
UM and its history are linked to the history of user-adaptive systems
Based on the way in which the UM updates its model of the user, the domain in which it is used, and the way the interface is caused to change
18 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling
For instance, UM + ratings = stereotype/probabilistic recommender system
UM + hypertext + adaptation rules = AHS UM + user goals + pedagogy + adaptation rules =
ITS UM representation, and how it learns about its
users tends to depend on the domain
19 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling Focusing on generic user modelling Has its roots in dialogue systems and philosophy
Need to model the participants to disambiguate referents, model the participants’ beliefs, etc.
Early systems (pre-mid-1985) had user modelling functionality embedded within other system functionality (e.g., Rich (recommendation system); Allen, Cohen & Perrault (dialogue processing))
20 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling
From 1985, user modelling functionality was performed in a separate module, but not to provide user modelling services to arbitrary systems
So one branch of user modelling focuses on user modelling shell systems
2001-UMUAI-kobsa (UM history).pdf
21 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling
Although UM has its roots in dialogue systems and philosophy, more progress has been made in non-natural language systems and interfaces (PontusJ.pdf)
GUMS (General User Modeling System) first to separate UM functionality from application - 1986 (Finin)
22 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling
GUMSAdaptive system developers can define
stereotype hierarchiesProlog facts describe stereotype membership
requirementsRules for reasoning about them
23 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling
At runtime:GUMS collects new facts about users using the
application systemVerifies consistencyInforms application of inconsistenciesAnswers application queries about assumptions
about the user
24 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling
Kobsa, 1990, coins “User Modeling Shell System”
UMT (Brajnik & Tasso, 1994): Truth maintenance systemUses stereotypesCan retract assumptions made about users
25 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling
BGP-MS (Kobsa & Pohl, 1995)Beliefs, Goals, and Plans - Maintenance SystemStereotypes, but stored and managed using
first-order predicate logic and terminological logic
Can be used as multi-user, multi-application network server
26 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling
Doppelgänger (Orwant, 1995)Info about user provided via multi-modal user
interfaceUser model that can be inspected and edited by
user
27 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling
TAGUS (Paiva & Self, 1995)Also has diagnostic subsystem and library of
misconceptionsPredicts user behaviour and self-diagnoses
unexpected behaviour
um (Kay, 1995)Uses attribute-value pairs to represent userStores evidence for its assumptions
28 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
History of User Modelling
From 1998 and with the popularisation of the Web, web personalisation grew in the areas of targeted advertising, product recommendations, personalised news, portals, adaptive hypertext systems, etc.
CSA3212: Topic 5© 2005- Chris Staff
29 of [email protected] University of Malta
Part II: UM Implementations
30 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
What might we store in a UM?
Personal characteristicsGeneral interests and preferencesProficienciesNon-cognitive abilitiesCurrent goals and plansSpecific beliefs and knowledgeBehavioural regularitiesPsychological statesContext of the interactionInteraction history
PontusJ.pdf, ijcai01-tutorial-jameson.pdf
31 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
From where might we get input? Self-reports on personal characteristics Self-reports on proficiencies and interests Evaluations of specific objects Responses to test items Naturally occurring actions Low-level measures of psychological states Low-level measures of context Vision and gaze tracking
ijcai01-tutorial-jameson.pdf
32 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Techniques for constructing UMs
Attribute-Value PairsMachine learning techniques & Bayesian
(probabilistic)Logic-based (e.g.inference techniques or
algorithms) Stereotype-based Inference rules
kules.pdf
33 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Attribute-Value Pairs
e.g., ah2002AHA.pdf The representation of the user and of the domain
are inextricably linked What we want to do is capture the “degree” to
which a user “knows” or is “interested” in some concept
We can then use simple or complex rules to update the UM and to adapt the interface
34 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Attribute-Value Pairs
Particularly useful for showing (simple) dependencies between concepts Complex ones harder to update
Can use IF-THEN-ELSE rules to trigger events Such as updating a user model Modifying the contents of a document (AHA!,
MetaDoc) Changing the visibility or viability of links
35 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Overview of AHA! Adaptive Hypertext for All! Each time user visits a page, a set of rules
determines how the user model is updated Inclusion rules determine the fragments in the
current page that will be displayed to the user (adaptive presentation)
Requirement rules change link colours to indicate the desirability of each link (adaptive navigation)
36 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Attribute-Value Pairs
From where do the attributes come?Need to be meaningful in the domain
(domain modelling)Can be concepts (conceptual modelling)Can be terms that occur in documents
(IR)
37 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Attribute-Value Pairs
What do values represent?Degrees of interest, knowledge,
familiarity, ...Skill level, proficiency, competenceFacts (usually as strings, rather than
numerical values)Truth or falsehood (boolean)
38 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Simple Bayesian Classifier
Rather than pre-determining which concepts, etc., to model, let features be selected based on observation
SBCs are also used in machine learning approaches to user modelingInstead of working with predetermined sets of
models, learn interests of current user
ProbUserModel.pdf
39 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Simple Bayesian Classifier Let’s say we want to determine if a document is
likely to be interesting to a user We need some prior examples of interesting and
non-interesting documents Automatically select document features
Usually terms of high frequency Assign boolean values to terms in vectors
To indicate presence in or absence from document
40 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Simple Bayesian Classifier
Now, for an arbitrary document, we want to determine the probability that the document is interesting to the user
P(classj | word1 & word2 & ... wordk)Assuming term independence, the
probability that an example belongs to classj is proportional to
P(class j ) P (wordi | class j )i
k
41 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Syskill & Webert
Learns simple Bayesian classifier from user interaction
User identifies his/her topic of interestAs user browses, rates web pages as “hot”
or “cold”S & W learns user’s interests to mark up
links, and to construct search engine query
webb-umuai-2001.pdf, ProbUserModel.pdf
42 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Syskill & Webert
Text is converted to feature vectors (term vectors) for SBC
Terms used are those identified as being “most informative” words in current set of pagesbased on the expected ability to classify
document if the word is absent from doc
43 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Simple Bayesian Classifier
Of course, the term independence assumption is unrealistic, but SBC still works well
Algorithm is fast, so can be used to update user model in real time
Can be modified to support ranking according to degree of probability, rather than binary
44 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Simple Bayesian Classifier
Needs to be “trained”, usually using small data sets
Works by multiplying probability estimates to obtain joint probabilitiesIf any is zero, results will be zero...Can use small constant (0.001) instead
(estimation bias) ...
45 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Personal WebWatcher
Predicting interesting hyperlinks from the set of documents visited by a user
Followed links are positive examples of user interests
Ignored links are negative examples of user interests
Use descriptions of hyperlinks as “shortened documents” rather than full docs
pwwTR.pdf
46 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Personal WebWatcher
Also uses a simple Bayesian classifier to recommend interesting links
where
TF(w, c) is term frequency of term w in document of class c (e.g., interesting/non-interesting), and TF(w, doc) is frequency of term w in document doc
P(c | doc) P(c)
w P (w |c)TF ( w,doc )
P (c i )w
P(w |c i )TF( w ,doc )
p(w |c) 1 TF (w,c)
# words TF (w i,c)i
47 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Personal WebWatcher
“Training” set is set of documents that user has seen and user could have seen but has ignored
Uses short description of document, rather than document vector itself
48 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Logic-based
Does a UM only contain facts about a user’s knowledge?
Can we also represent assumptions, and assumptions about beliefs?
Assumptions are contextualised, and represented using modal logic (AT:ac, or assumption type:assumption content)
pohl1999-logic-based.pdf
49 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Logic-based
We can also partition assumptions about the user
50 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Logic-based
Advantage is that beliefs, assumptions, facts are already in logical representation
Makes it easier to draw conclusions about the user from the stored knowledge
51 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Stereotype-based
Originally proposed by Rich in 1979Captures default information about groups
of usersTends not to be used anymore
1993-aui-kobsa.pdf
52 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Stereotype-based
Kobsa points out that developer of stereotypes needs to fulfil three tasks Identify user subgroups Identify key characteristics of typical user in subgroup
So that new user may be automatically classified
Represent hierarchically ordered stereotypesFine-grained vs. coarse-grained
53 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Inference rules e.g., C-Tutor, avanti.pdf May use production rules to make inferences
about user Also, to update system about changes in user state
or user knowledge Note that Pohl points out that all user models (that
learn about the user) must infer assumptions about the user (pohl1999-logic-based.pdf)
54 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Adaptive Hypertext Systems
“[B]y adaptive hypermedia we mean all hypertext and hypermedia systems which
reflect some features of the user in the user model and apply this model to adapt various
visible aspects of the system to the user”Brusilovsky, P. (1996). Methods and techniques of adaptive hypermedia, in User Modeling
and User-Adapted Interaction 6 (2-3), pp. 87-129. Available on-line at: http://www.contrib.andrew.cmu.edu/~plb/UMUAI.ps
55 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Adapted from Horgen, S.A., 2002, "A Domain Model for an Adaptive Hypertext System based on HTML", MSc Thesis, Chapter 4 (Adaptivity), pg. 32. Available on-line from http://www.aitel.hist.no/~svendah/ahs.html (iui.pdf)
56 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Conclusion
User Models can represent user beliefs, preferences, interests, proficiencies, attitudes, goals
User models are used in AHS to modify hyperspace In IR to select better (more relevant) documents
More likely to use analytical cognitive model, but can still use “simple” models
CSA3212: Topic 5© 2005- Chris Staff
57 of [email protected] University of Malta
Part III: Types of UM
58 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Types of User Models
User Models have their roots in philosophy and learning
Student model assumed to be some subset of the knowledge about the domain to be learnt
Consequently, the types of user model have been heavily influenced by this
59 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Student Models Student Models are used, e.g., in Intelligent
Tutoring Systems (ITSs) In ITS we know user goals, and may be able to
identify user plans The domain/expert’s knowledge must be well
understood Assumption that user wants to acquire expert’s
knowledge Plan means moving from user’s current state to
state that user wants to achieve
60 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Student ModelsIf we assume that expert’s knowledge is
transferable to student, then student’s knowledge includes some of the expert’s knowledge
Overlay, differential, perturbation models (from neena_albi_honours.pdf p25-)
61 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Overlay Models
SCHOLAR (Carbonell, 1970)Simplest of the student modelsStudent knowledge (K) is a subset of
expert’sAssumes that K missing from student model
is not known by the studentBut what if student has incorrectly learnt K?
62 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Overlay Models
Good when subject matter can be represented as prerequisite hierarchy
K remaining to be acquired by student is exactly difference between expert K and student K
Cannot represent/infer student misconceptions
63 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Differential Models
WEST (Burton & Brown, 1989)Compares student/expert performance in
execution of current taskDivides K into K the student should know
(because it has already been presented) and K the student cannot be expected to know (yet)
64 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Differential Models
Still assumes that student’s K is subset of expert’s
But can differentiate between K that has been presented but not understood and K that has not yet been presented
65 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Perturbation Models
LMS (Sleeman & Smith, 1981)Combines overlay model with
representation of faulty knowledgeBug library
Attempts to understand why student failed to complete task correctly
Permits student model to contain K not present in expert’s K
CSA3212: Topic 5© 2005- Chris Staff
66 of [email protected] University of Malta
Part IV: Understanding user behaviour
67 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Making assumptions about users
Browsing behaviourWhat does a user’s browsing behaviour tell us
about the user?
68 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Making assumptions about users
Searle (1969)... when a speech act is performed certain presuppositions must have been valid for the speaker to perform the speech act correctly (from 1995-UMUAI-kobsa.pdf, 1995-COOP95-kobsa.pdf)
69 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Making assumptions about users
If the user requests an explanation, a graphic, an example or a glossary definition for a hotword, then he is assumed to be unfamiliar with this hotword.
1996-kobsa.pdf
70 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Making assumptions about users
If the user unselects an explanation, a graphic, an example or a glossary definition for a hotword, then he is assumed to be familiar with this hotword.
1996-kobsa.pdf
71 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Making assumptions about users
If the user requests additional details for a hotword, then he is assumed to be familiar with this hotword.
1996-kobsa.pdf
72 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
User Actions in Hypertext Actions that can be performed in hypertext
Follow link Don’t follow link Print Bookmark Go to bookmark Backup Go to URL ...
73 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Understanding Browsing Behaviour
What might each of these actions mean?Can we relate them to Kobsa’s
assumptions?Do we need link analysis first?
74 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Identifying Browsing Behaviour
Lost in Hyperspace (otter2000.pdf)Honing in on informationNeeding more help/informationBeing un/familiar with topic/web spaceInterested in topicUninterested in topicChanging topic
75 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Identifying Browsing Behaviour
Search browsing“directed search; where the goal is known”
General Purpose Browsing“consulting sources that have a high likelihood
of items of interest”
The serendipitous user“purely random”
catledge95.pdf
76 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Understanding Browsing Behaviour
How can understanding browsing behaviour help us create better adaptive hypertext systems?Less intrusiveJust-in-Time supportDon’t give more info when it is not
required/wantedEfficient use of resources
77 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Conclusions
The ability to model the user allows reasoning about the user to tailor an interaction to the user’s needs and requirements...
... especially when the user is unable to describe what it is they need
Tightly bound to domain/expert knowledge
78 of [email protected] University of Malta
CSA3212: Topic 5© 2005- Chris Staff University of Malta
Conclusions
Significant efforts to decouple the user model from the application
May be too expensive to accurately model all domains, and in any case, goal of many adaptive systems is not to help user become expert, but to provide timely assistance at the right level of detail