improving explicit preference entry by visualising data similarities
Post on 12-Jan-2015
535 Views
Preview:
DESCRIPTION
TRANSCRIPT
improving explicit preference entry by visualising data similaritieskris jack and florence duclaye
13 january 2008
kris jack – p2
the problem in context
background
proposed solution
user evaluation
results
discussion
conclusion
1
summary
2
3
4
5
7
6
kris jack – p3
1the problem in context
kris jack – p4
context
general context recommender systems attempt to offer items to users that will be appreciated the quality of these recommendations is largely constrained by the data
• a user’s preferences
• domain-specific ‘general knowledge'• the items that can be recommended• previous users’ opinions of items
the acquisition of such data is of central importance
specific context a hybrid content-based and collaborative filtering recommendation system the system will be implemented in the domain of cinema the user can explicit enter their preferences
kris jack – p5
the problem
entering your preferences explicitly can be boring can be difficult
if a system is too much trouble to use, then it simply won't be used
how can we improve the explicit preference entry process?
maximise the number of explicit preferences that could be elicited within a given period of time
kris jack – p6
2background
kris jack – p7
definitions
preferences, many types and definitions exist concentrating on the elicitation of monadic preferences represent a user’s like, dislike, or indifference towards an item or item
attribute (e.g. “i like tim burton”, “i dislike horror movies” and “i love kill bill”)
preference acquisition strategies explicit – user explicitly enters their preferences implicit – learning strategies that are non-invasive (e.g. user-profiling
and collaborative filtering)
explicit preference entry (epe) interface an interface that asks users to explicitly give their preferences
towards items and item attributes (e.g. “i love comedies”)
kris jack – p8
some existing epe interfaces
recommenders with epe interfaces minekey (www.minekey.com) stumbleupon (www.stumbleupon.com) movielens
drawbacks boring to use difficult to be inspired
(when guidance is lacking) difficult to describe yourself
in their terms at times
stumbleupon
minekey
kris jack – p9
data visualisation
perhaps we can improve epe interfaces by visualising data data visualisation techniques have had some success in
recommenders music plasma (www.musicplasma.com) amaznode (amaznode.fladdict.net/)
music plasma amaznode
kris jack – p10
3proposed solution
kris jack – p11
visualising data similarity in an epe interface
the epe interface should encourage users to enter their preferences may guide users based upon the preferences required should be enjoyable to use and not boring
the epe interface creation process must be robust and reliable desirable if it is automated from start to finish
creation process summary input a list of items or item descriptors (e.g. actors) using a similarity metric, find the similarities between data elements visualise the similarities between the data elements
kris jack – p12
instantiating the system's semantic knowledge
the system's semantic knowledge describes the similarity between descriptors in the database
the notion of similarity is necessarily subjective
how similar do you find these two actors?
(robert de niro and al pacino)
kris jack – p13
strategies for defining semantic similarities considered
instantiation by hand differencing mechanism co-occurrence measures clustering algorithms collaborative filtering techniques
opted for one based on co-occurrences of actor names found using the google search estimates
where m is the total number of pages considered, f(i1) and f(i2) are the number of hits for i1 and i2 respectively, and f(i1,i2) is the number of hits for the co-occurrence of i1 and i2.
in essence, the more often two items appear together on the same web pages, the more similar they are
the measure of semantic relatedness server provides free access
1 2 1 21 2
1 2
max{log ( ),log ( )} log ( , )( , )
log min{log ( ),log ( )}
f i f i f i id i i
M f i f i
−=−
kris jack – p14
normalised google distance
actors jackie chan bruce lee jane fonda
jackie chan
2,420,000 (0.0)
965,000(0.09)
145,000(0.26)
brucelee
965,000(0.09)
2,630,000(0.0)
46,700(0.37)
jane fonda
145,000(0.26)
46,700(0.37)
1,930,000(0.0)
the number of google hits for actor pairs and their normalized google distances given in brackets (the smaller the distance, the more similar the actors)
kris jack – p15
data visualisation
use of the radial tree layout to visualise data similarities manageable linear complexity in laying out the tree
• efficient even with several hundreds of nodes the more similar two items are, the closer their proximity in the graph a radial layout encourages users to explore the tree in a less hierarchical
fashion as it is unclear where the root node is tree can be focussed upon any node in the tree (implementing a smooth
transition animation)• users have previously found this form of visualisation attractive
implementation available in prefuse (www.prefuse.com) library
strategy each item (actor) is represented by a node in the tree connect every node with their two closest nodes (using item similarity)
kris jack – p16
radialtree
kris jack – p17
radial tree (partial)
kris jack – p18
epe interface
the visualisation is mounted in an epe interface that allows users to zoom in by right clicking on a graph area (in zoomed out mode) and zoom
out by right clicking on a graph area that does not contain an actor (in zoomed in mode);
pan within the graph by left clicking on a graph area and dragging in the direction to pan;
search for an actor by typing the actor’s name in the search box. when the user starts to type a name, the mode changes to zoomed out mode and all actor nodes who’s names match the string are enlarged;
change the preference towards an actor (like, dislike, neutral, no preference) by righting clicking on the actor’s node (in zoomed in mode);
re-organise the graph to centre upon one actor by double left clicking on another actor node.
kris jack – p19
epe interface
kris jack – p20
kris jack – p21
4user evaluation
kris jack – p22
evaluating the epe interface
evaulated: choice of similarity metric in the context of actors type of preferences elicited epe interface's ease of use appreciation of the epe interface
materials epe interface mounted with 3 different graphs:
• organised graph (nodes positions according to actor similarity)• unorganised graph (organised graph with nodes randomised)• demonstration graph (organised graph with nodes randomised)
each graph contained the same 500 nodes (most frequent actors from an in-house french database)
instruction sheet
kris jack – p23
participants and procedure
28 participants (14 male, 14 female) procedure
practice the functions of the epe interface using the demonstration graph (took 10 minutes on average)
task• enter as many actor-based preferences as possible in 5 minutes• once with the organised graph and once with the unorganised graph
(following a within-subjects design with 2 groups of 14 participants)• note that the graphs were not named here, the tasks were referred to as
task 1 and task 2 participants completed a questionnaire on terminating the tasks
kris jack – p24
hypothesis and measurements
hypothesis participants will find it easier to declare their preferences for actors in
the organised graph task than in the unorganised graph task, within the same time period
measuring the ease of declaring preferences quantity of preferences entered subjective questioning in the questionnaire
kris jack – p25
5results
kris jack – p26
preference elicitation
more preferences were entered using the organised graph: significant increase in 'like' preferences (34%) decrease in 'dislike' and 'neutral' preferences
Preference Elicitation
0
10
20
30
40
50
60
All Like Dislike Neutral
Organised
Unorganised
kris jack – p27
perceived ease of entering preferences
participants reported that it was: easy to enter preferences using the organised graph neither easy nor difficult to enter preferences using the unorganised
graph
ease of preference entry statements
mean
organised unorganised
"i found it easy to enter my preferences." 3.96 3.36
"entering my preferences demanded too much effort" (mean reversed) 3.64 3.11
cronbach’s alpha = 0.80
kris jack – p28
perceived differences between graphs
22 participants (79%) reported differences between the two graphs
commented that the organised graph had been hand designed so that it was easier to navigate; rearranged itself based on the actors that the participant said that they liked; arranged the participant’s favourite actors together; had more connections between nodes; arranged actors together who:
• co-starred in the same films;• shared the same nationality;• shared the same degree of celebrity;• were similar to one another.
commented that the unorganised graph was less organised
kris jack – p29
appreciation of the epe interface
participants enjoyed using the interface and would be happy using it again (mean = 4.11/5.00, sd = 0.99)
suggested improvements preference changing. some participants did not like having to click
twice to register a dislike and three times to enter a neutral preference.
zooming in. some participants would have preferred a precise indication of the region into which they could zoom into before zooming.
zooming out. some participants felt lost when zoomed in on the actors as they were not sure of where they were with respect to the entire map
kris jack – p30
6discussion
kris jack – p31
discussion
the participants mark more 'like' preferences with the organised graph (34%)
find more actors who they like and less who they dislike or are neutral towards
why?• in searching, participants tended to begin by using the search feature,
then zoom in on their desired actor
• when at the zoomed in level, they would pan around to find other actors.
• actors who were in close proximity tended to be similar
• similar actors tend to be liked too applications that can exploit likes better than dislikes may want to
introduce semantic similarity in an epe interface
kris jack – p32
discussion
how well did the notion of similarity come across? with only 5 minutes of exposure to each graph, the majority of
participants found that one was organised and that the other was lesser so or not at all
the word similar was repeatedly used by participants results serve to validate the use of the google distance metric in this
area the similarity metric thus goes some way to replacing what is
traditionally in the domain of human-design decisions the epe interface was very much appreciated
participants liked it and wanted to use it again they commented that it was more like playing a game and not like
entering their preferences
kris jack – p33
discussion
participants report that the organised graph is easier to use an organised graph is easier to navigate than an unorganised
graph they find more actors who they like
addressing interface issues replace the right click to change a preference with three icons next to
the node. a single click on the item will designate the corresponding preference
offer a 'mini-map', with the absolute position of the main map indicated, that is always zoomed out
kris jack – p34
discussion
when should the epe interface appear in the recommendation process?
from the start, users should be able to use it• initial entry of preferences
all throughout the recommendation process also• could be used to visualise learned or predicted preferences too, allowing the user to
correct any mistakes at a visual level
what other benefits does a similarity-based epe interface bring? users become aware of the notion of similarity as used by the system the logic of the system, in this case the positioning of actor nodes, becomes
learned imagine a system that uses this form of similarity to produce non-exact results
for searches (e.g. cannot fine any jackie chan films, would you like bruce lee films instead?)
understanding the logic of the system is very important in developing trust in the system
kris jack – p35
7conclusion
kris jack – p36
conclusion
a new epe interface is introduced that can takes data and organises it based on a robust similarity metric
data similarities are visualised into a pleasing tree-based graph users can navigate through the graph and explicitly enter their
preferences for different items interface favours elicitation of 'like' preferences
users enter 34% more 'like' preferences when the graph is organised with the similarity metric compared to when it is left unorganised
users report a reduction in cognitive effort when using the organised graph
the epe creation process is a robust and flexible solution to eliciting explicit user preferences in a recommendation system
kris jack – p37
the end
many thanks for your attention
top related