eric horvitz, jack breese, david heckerman, eric horvitz, jack breese, david heckerman, david hovel,...
Post on 19-Dec-2015
225 views
TRANSCRIPT
Eric Horvitz, Jack Breese, David Heckerman, Eric Horvitz, Jack Breese, David Heckerman,
David Hovel, Koos RommelseDavid Hovel, Koos Rommelse
Microsoft Research Redmond, WA 98052Microsoft Research Redmond, WA 98052
Presented by Suman B. Pakala, Feng XuPresented by Suman B. Pakala, Feng XuCSCE 582 Fall 2002CSCE 582 Fall 2002
Instructor: Marco ValtortaInstructor: Marco Valtorta
The LumiThe Lumièère Project: Bayesian User re Project: Bayesian User Modeling for Inferring the Goals and Modeling for Inferring the Goals and
Needs of Software Users.Needs of Software Users.
IntroductionIntroduction
LumiLumièère started in 1993. Components re started in 1993. Components used in office 95, 97.used in office 95, 97.Constructing Bayesian user models for Constructing Bayesian user models for reasoning reasoning Gaining access to a stream of events from Gaining access to a stream of events from software applicationssoftware applicationsDeveloping a languageDeveloping a languageDeveloping persistent user profileDeveloping persistent user profileDevelopment of an overall architectureDevelopment of an overall architecture
Bayesian user modelingBayesian user modeling
Modeling the beliefs, intentions, goals and Modeling the beliefs, intentions, goals and needs of users.needs of users.
Goals are tasks or subtasks.Goals are tasks or subtasks.
Needs are either information or automated Needs are either information or automated actions.actions.
Influence Diagram
An influence diagram for providing intelligent assistance given uncertainty in a User’s background, goals and competency in working with a software application
Classes of evidenceClasses of evidence
SearchSearch
Focus of attentionFocus of attention
IntrospectionIntrospection
Undesired effectsUndesired effects
Inefficient command sequenceInefficient command sequence
Domain-specific syntactic and semantic Domain-specific syntactic and semantic contentcontent
A portion of a Bayesian user Model for inferring
A Model used for inferring the likelihood of the user needing assistance,considering profile information as well as observations of recent activity
Partial Bayesian user model for Partial Bayesian user model for ExcelExcel
Markov Model forMarkov Model for Temporal ReasoningTemporal Reasoning
Markov model for temporal reasoning assuming dependencies among the goalsOf a user in adjacent time periods. A persistent Profile variable influencesGoals and observations in all periods.
Temporal ReasoningTemporal Reasoning
Lag of event from present moment
Formulation of the temporal reasoning problem as a set of single-stageproblems. We directly assess conditional probabilities of actions as afunction of time that passed since actions occurred.
P(E_i_tn | Goal_t0) P(E_i_t-1 | Goal_t0) P(E_i_t0 | Goal_t0)
Bridging the gap between System Events and Bridging the gap between System Events and User ActionsUser Actions
User’s Actions that can be accessed:User’s Actions that can be accessed:Mouse and keyboard actionsMouse and keyboard actionsStatus of data structures in Excel filesStatus of data structures in Excel filesMenus being visitedMenus being visitedDialog boxes being opened and closedDialog boxes being opened and closedSelection of specific objects like charts, etc.Selection of specific objects like charts, etc.
They are modified into System events and They are modified into System events and modeled as:modeled as:Menu SurfingMenu SurfingMouse MeanderingMouse MeanderingMenu jitter, etc.Menu jitter, etc.
LumiLumièère Events Languagere Events Language
Why? – To make modeling flexibleWhy? – To make modeling flexiblePrimitives:Primitives: Rate(Rate(xi,txi,t)) One of(One of({{x1, ….,xnx1, ….,xn},},tt)) AllAll(({{x1, ….,xnx1, ….,xn},},tt)) SeqSeq((x1, ….,xn,tx1, ….,xn,t)) Dwell(Dwell(tt)) Example of an event: User dwelled for atleastExample of an event: User dwelled for atleast
tt seconds, following a scroll seconds, following a scroll
A high level architectural view of Lumière/ Excel
Architecture & Control PoliciesArchitecture & Control Policies
Architecture:Architecture:Events => Time stamped observationsEvents => Time stamped observationsObservations => Bayesian Model => P(user needs)Observations => Bayesian Model => P(user needs)If Query, P(events) + Bayesian term spotting => PP(needs)If Query, P(events) + Bayesian term spotting => PP(needs)Also, L(needs) => Control(automated assistance)Also, L(needs) => Control(automated assistance)
Control Policies:Control Policies:Pulsed strategyPulsed strategyEvent-driven control policy + trigger eventsEvent-driven control policy + trigger eventsAugmented pulsed approachAugmented pulsed approachDeferred analysisDeferred analysis
LumiLumièère / Excel In Operationre / Excel In Operation
Atomic events stream, Probability distribution over needs, AssistanceMonitoring agent, User interface for the prototype
When a query is made:When a query is made:
(a) (b)
(a) Inference based on actions. (b) Revised distribution after query is made
Autonomous display of assistanceAutonomous display of assistance
Actual application, Assistance monitoring agent, Offer of assistance
Beyond Real time assistanceBeyond Real time assistance
Information that is recommended for the user to review offline
LumiLumièère in the real world: Office Assistantre in the real world: Office Assistant
Broader but shallower models Broader but shallower models (compared with Lumi(compared with Lumièère/Excel)re/Excel)
Rich set of context variablesRich set of context variables
No persistent user profiling, No competency reasoningNo persistent user profiling, No competency reasoning
Small set of relatively atomic user actionsSmall set of relatively atomic user actions
Only the most recent events are consideredOnly the most recent events are considered
If words available, context and recent actions are not If words available, context and recent actions are not consideredconsidered
Inference results are available only when user requests Inference results are available only when user requests
(Autonomous assistance not employed)(Autonomous assistance not employed)
Current ResearchCurrent Research
Learning Bayesian models from user log dataLearning Bayesian models from user log data
Integrating new sources of eventsIntegrating new sources of events
Automated dialog for users to express goals and needsAutomated dialog for users to express goals and needs
Integrating vision and gaze trackingIntegrating vision and gaze tracking
Use of Value of Information computations Use of Value of Information computations (to engage user (to engage user in dialog to access costly information about activity and program in dialog to access costly information about activity and program state)state)