technology to support individuals with cognitive impairment martha e. pollack computer science &...
TRANSCRIPT
![Page 1: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/1.jpg)
Technology to Support Individuals with Cognitive Impairment
Martha E. PollackComputer Science &
EngineeringUniversity of Michigan
![Page 2: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/2.jpg)
Autominder
• Model, update, and maintain the client’s plan– Including complex temporal and causal constraints
• Monitor the client’s performance– Updating the plan as execution proceeds
• Reason about what reminders to issue, and when– To most effectively ensure compliance, without sacrificing
client independence
![Page 3: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/3.jpg)
Client Client ModelerModeler
Plan Plan ManagerManager
IntelligentIntelligentReminderReminderGeneratorGenerator
ClientPlan
Activity Info
Inferred Activity
Sensor Data
Reminders
Client Model Info
Activity Info
Preferences
Plan Updates
ClientModel
Autominder ArchitectureWhat should the client do?
Technologies: Automated Planning, Constraint-Based Temporal Reasoning
What is the client doing?
Technologies: Dynamic Bayesian InferenceIs a reminder needed?
Technologies: Iterative Refinement Planning, Reinforcement Learning
![Page 4: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/4.jpg)
Autominder Example
Req/Opt Activity Allowed Expected Observed
R toilet use 10:45-11:05
R lunch 12:00-12:45
O TV 14:00-14:30
10:55
R toilet use13:55-14:15
REMIND 12:25
REMIND 13:55
12:28
![Page 5: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/5.jpg)
Robot Platform
• Nomadic Technologies Scout II
w/custom-designed head
– Multiple sensors: lasers, sonars, microphone, touchscreen, camera vision, wireless ethernet
– Effectors: motion, speakers, display screen, facial expression
“Pearl”[courtesy Carnegie MellonUniv. Robotics Institute]
![Page 6: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/6.jpg)
“Ubicomp” Platform
• Handheld or wearable device– Currently: HP iPaq
• Deployed in a “smart” environment with multiple sensors (ubiquitous computing environment)
![Page 7: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/7.jpg)
The Plan Manager
• Maintains up-to-date record of client’s planned activities and their execution status– Eating– Hydrating – Toileting– Medicine-taking – Exercise – Social activities – Doctors’ appointments– etc.
![Page 8: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/8.jpg)
How Does it Work?
• Models constraints on future actions– Lunch takes between 25 and 35 minutes – Take meds within one hour of finishing lunch – Watch the news at either 6pm or at 11pm
• Performs efficient constraint processing when key events occur:– New planned activity added.– Existing activity modified or deleted.– Planned activity performed.– Critical time bounds passed.
![Page 9: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/9.jpg)
Small Example
ClientPlan
1. New Activity2. Mod/Deletion3. Activity Execution4. Passed Time Bound
PLAN MANAGER
:0 MS – LE :60“Take meds within 1 hour of lunch”
LE = 12:15“Lunch ended at 12:15”-----------------------------12:15 MS 13:15“Take meds by 1:15”
![Page 10: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/10.jpg)
Temporal Reasoning in AI
An important task & exciting research topic, otherwise we would not be here
• Temporal Logic• Temporal Networks
– Qualitative relations: • Before, after, during, etc.
• interval algebra, point algebra
– Quantitative/metric relations: • 10 min before, during 15 min, etc.
• Simple TP (STP), Temporal CSP (TCSP), Disjunctive TP (DTP)
![Page 11: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/11.jpg)
Temporal Network: example
Tom has class at 8:00 a.m. Today, he gets up between 7:30 and 7:40 a.m. He prepares his breakfast (10-15 min). After breakfast (5-10 min), he goes to school by car (20-30 min). Will he be on time for class?
![Page 12: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/12.jpg)
Simple Temporal Network (STP)
• Variable: Time point for an event
• Domain: A set of real numbers (time instants)• Constraint: An edge between time points ([5, 10] 5Pb-Pa10)
• Algorithm: Floyd-Warshall, polynomial time
![Page 13: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/13.jpg)
Example
A
B
C
[10 15]
[5 10]
?
![Page 14: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/14.jpg)
Example
A
B
C
15
10
-10
-5
![Page 15: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/15.jpg)
Example
A
B
C
15
10
-10
-510
![Page 16: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/16.jpg)
Example
A
B
C
15
10
-10
-510 0
![Page 17: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/17.jpg)
Example
A
B
C
[10 15]
[5 10]
[0 10]
![Page 18: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/18.jpg)
Other Temporal Problems
Temporal CSP: Each edge is a disjunction of intervals
STP TCSP
Disjunctive Temporal Problem: Each constraint is a disjunction of edges
STP TCSP DTP
![Page 19: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/19.jpg)
Search to solve the TCSP/DTP
• TCSP [Dechter] and DTP [Stergiou & Koubarakis] are NP-hard• They are solved with backtrack search• Every node in the search tree is an STP to be solved• An exponential number of STPs to be solved
![Page 20: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/20.jpg)
CM: Client Modeler
Given what can be observed• Sensor input: client moved to kitchen • Clock time: at 7:23 a.m.• Client plan: breakfast should be eaten between 7 and 8• Model of previous actions: client has not yet eaten breakfast• Learned patterns: 82% of the time, client starts breakfast between 7:10 and 7:25• Reminder information: we issued a reminder at 7:21
Infers what has been done• Client Activity: probability that client has begun breakfast
![Page 21: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/21.jpg)
How Does it Work?
• Models probabilistic relations among observations and actions
• Performs Bayesian update, extended to handle temporal relations• Asks for confirmation when needed!
started
breakfast
breakfastreminder issued
went tokitchen
reminder kitchen start-breakfast Y Y .95 Y N .10 N Y .8 N N .03
![Page 22: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/22.jpg)
Intelligent Reminders
• Decides whether and when to issue reminders• Given a client’s plan and its execution status:
– Easy to generate reminders• Remind at earliest possible time of each action
– Harder to “remind well”• Maximize likelihood of appropriate performance of
ADLs and other key activities• Facilitate efficient performance• Avoid annoying client• Avoid making client overly reliant
![Page 23: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/23.jpg)
How Does it Work?
LB D
TV
Midnight
8:00 16:0012:00
12:00
LB D
TV
Midnight
8:00 16:0012:00
12:00
LB D
TV
Midnight
8:30 16:0012:00
12:00
8:30 12:32• Initially: schedule reminders for earliest possible time• Apply “rewrite rules” to improve remders:
• Used preferred times for reminders• Combine “near” reminders that are compatible
• e.g.: “drink water” and “take pills”• Reschedule reminders for conflicting activities
![Page 24: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/24.jpg)
Current Status of Autominder
• V.0 (Autominder + Pearl) field-tested for client acceptability on Pearl at Longwood Elderly Care Facility in Oakmont, PA, summer, 2001
• V.1 of Autominder implemented – Java, Lisp on Wintel machines
• Data collection with three Oakmont residents completed summer 2002; with Ann Arbor TBI patient summer 2003
![Page 25: Technology to Support Individuals with Cognitive Impairment Martha E. Pollack Computer Science & Engineering University of Michigan](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb95503460f9497fdf5/html5/thumbnails/25.jpg)
Key Challenges for Cognitive Orthotics
• Technological– Advanced AI Techniques
– HCI
– Sensor Networks for Inference of Daily Activities
– Mechanisms to Ensure Privacy and Security
• Policy– Mechanisms to Ensure Privacy
– Reimbursement Policies