hfes 2016, human factors and ergonomics in...
TRANSCRIPT
Hugh Dubberly, Dubberly Design OfficeStephen B. Wilcox, Design Science
HFES 2016, Human Factors and Ergonomics in Healthcare: Shaping the FutureSan Diego April 15, 2016
From Devices to PlatformsImplications of the Internet of Thingsfor Healthcare
2Dubberly Design Office · From Devices to Platforms · April 15, 2016
...we are in the middle of a dramatic and broad technological and economic shift in which software companies are poised to take over large swathes of the economy...
Health care and education, in my view, are next up for fundamental software-based transformation.”
— Marc Andreessen, founder, Netscape and Andreessen-Horowitz
Wall Street Journal OpEd www.wsj.com/articles/SB10001424053111903480904576512250915629460
“...software is eating the world.
3Dubberly Design Office · From Devices to Platforms · April 15, 2016
“If you went to bed last night thinking you’re an industrial company, you’re going to wake up this morning as a software and analytics company.”
— Jeff Immelt, Chairman and CEO, General Electric
Minds + Machines 2014www.ge.com/stories/industrial-internet
1981 1995 2016Personal Computers The Internet Internet of Things (IoT)
Social
Mobile
Voice UIs
SaaSDrones Robots
Virtual realityAugmented reality
Big dataLinked dataOpen data
GlassesGogglesWearables
AI 2.0Machine learningPredictive analytics
CNC3D printing
Cloud computingon-demand
4Dubberly Design Office · From Devices to Platforms · April 15, 2016
Technological change comes in waves,which interact to create “combinatorial innovation.”[1]
– Work has gone digital. – People got connected. – Now, things are connecting, too.
[1] Hal Varian
5Dubberly Design Office · From Devices to Platforms · April 15, 2016
2016 is like 1981 and 1995 all over again.You can see the next wave coming.
It goes by many names: – Internet of Things (IoT) – Internet of Everything, Cisco – Industrial Internet, GE – Smarter Planet, IBM – Living Services, Accenture – Platform World, Sapient.Publicis – Social CRM or Social Business – Digital Engagement – Digital Transformation
6Dubberly Design Office · From Devices to Platforms · April 15, 2016
The change has already begun,for example
Amazon announced AWS IoT, also testing pop-up stores in malls featuring IoT devices and Echo.
GE hired 2,000 engineers in San Ramon, builds Predix platform, GE Digital now has 30,000 employees.
Sears opens a series of “Connected Home” showcase stores.
Target launches “Open House” IoT showcase store in San Francisco.
Fixedcommunications
Mobilecommunications
Computers
Consumer electronicsand medical devices
Industrial devices(military and aerospace)
Automotive
2013 2014 2015 2016 2017 2018 2019 2020
10b
20b
30b
40b
50b
7Dubberly Design Office · From Devices to Platforms · April 15, 2016
By 2020, ~50 billion devices will be connected to the Internet;today, ~7 billion computers and tablets are connected.
Sources: The Economist and Cisco
Product, Service, and Ecologies Diagrams
Product Service Ecologies
Product Service Product Service
8Dubberly Design Office · From Devices to Platforms · April 15, 2016
Products no longer stand alone.Increasingly, they exist in complex service webs.
[1] Source: Automotive Sensors and Electronics Expo 2015http://www.automotivesensors2015.com/
The average vehicle includes 60–100 sensors;that figure may grow to 200 by 2020. [1]
[2] Source: The New York Timeshttp://www.nytimes.com/2010/02/05/technology/05electronics.html
In 2010, basic cars included 30 microprocessors;luxury cars had as many as 100. [2]
1. Product2. Smart product
Software
9Dubberly Design Office · From Devices to Platforms · April 15, 2016
Products are becoming “smart”.Product
+ Sensor + Computer = Smart Product
—Michael Porter and James Heppelmann, How Smart, Connected Products Are Transforming CompetitionHarvard Business Review, November 2014https://hbr.org/2014/11/how-smart-connected-products-are-transforming-competition
1. Product2. Smart product
Software
3. Smart, connected product
Remotedevices
Software
10Dubberly Design Office · From Devices to Platforms · April 15, 2016
Sets of smart products are connecting.Smart Product
+ Network + Cloud Service = Smart, Connected Product
—Michael Porter and James Heppelmann, How Smart, Connected Products Are Transforming CompetitionHarvard Business Review, November 2014https://hbr.org/2014/11/how-smart-connected-products-are-transforming-competition
1. Product2. Smart product
Software
3. Smart, connected product
Remotedevices
Software
4. Product system
Planters
Combineharvesters
Tillers
Tractors
FarmEquipment
System
Remotedevices
Software
11Dubberly Design Office · From Devices to Platforms · April 15, 2016
Sets of connected products form product systems.Smart, Connected Product
+ other Smart, Connected Products = Product System
—Michael Porter and James Heppelmann, How Smart, Connected Products Are Transforming CompetitionHarvard Business Review, November 2014https://hbr.org/2014/11/how-smart-connected-products-are-transforming-competition
1. Product2. Smart product
Software
3. Smart, connected product
Remotedevices
Software
4. Product system
Planters
Combineharvesters
Tillers
Tractors
FarmEquipment
System
Remotedevices
Software
5. Systems of systems
IrrigationSystem
Irrigation nodes
Irrigationapplication
Fieldsensors
Weather DataSystem
Weather dataapplication
Weather forecastsWeather maps
Rain, humidity,temperature sensors
FarmManagement
System
SeedOptimization
SystemSeed database
Farm performancedatabase
Seed optimizationapplication
Planters
Combineharvesters
Tillers
Tractors
FarmEquipment
System
Remotedevices
Software
12Dubberly Design Office · From Devices to Platforms · April 15, 2016
Systems connect to other systems, forming ecologies.Product Systems
+ other Product Systems= Product-Services Ecology
—Michael Porter and James Heppelmann, How Smart, Connected Products Are Transforming CompetitionHarvard Business Review, November 2014https://hbr.org/2014/11/how-smart-connected-products-are-transforming-competition
13Dubberly Design Office · From Devices to Platforms · April 15, 2016
Farms are becoming automated factories.Plants are attached to sensors, connected to networks, generating data.
14Dubberly Design Office · From Devices to Platforms · April 15, 2016
Macro view: processed satellite images of crop growth over time,e.g., central Iowa, March 29 to October 23, in 8 day increments.Algorithms automatically align images,remove clouds,and detect vegetation.
15Dubberly Design Office · From Devices to Platforms · April 15, 2016
Daily weather data can augment machine learning.Precipitation,temperature,wind direction and speed,snow cover,and cloud covercan aid forecasting.
Music Listener iPod
Hardware Software Networked Service Marketplace
iTunes iTunes Store Record Companies Artists
MENU
16Dubberly Design Office · From Devices to Platforms · April 15, 2016
iPod was the proto-IoT device—an integrated systemof hardware, software, and networked services.
Ride Seeker Driver
Database
Matching Service
GPS
MapsMembers
Drivers
CarPhone Phone
17Dubberly Design Office · From Devices to Platforms · April 15, 2016
VCs are no longer funding stand-alone apps;Uber isn’t just an app; it’s an IoT platform for logistics.
Customerdatabase
Batteries
Microprocessor Cloud
ConnectedMobile App
Speaker
AImachine-learningsystem
Microphone
Wi-Fi
18Dubberly Design Office · From Devices to Platforms · April 15, 2016
Last fall, Mattel relaunched Barbie, as a smart, connected product.Barbie + sensor + computer + network + cloud services = Eliza 2016
– Recognizes you and what you say – Learns about you – Converses with you,
using 8,000 pre-recorded phrases – Extends up to 120 exchanges
https://en.wikipedia.org/wiki/ELIZAhttp://www.nytimes.com/2015/09/20/magazine/barbie-wants-to-get-to-know-your-child.htmlhttp://www.nytimes.com/2015/10/16/business/mattel-aims-to-reanimate-sales-with-interactive-barbie.html
Business
Marketing
Mar-com
CRM MMDB
Member MarketingDatabase
Business
New products + servicesbuilt on the MMDB.
Marketing
Mar-com
CRM
MMDB
Business
Marketing
Mar-com
CRM MMDB
Member MarketingDatabase
Business
New products + servicesbuilt on the MMDB.
Marketing
Mar-com
CRM
MMDB
19Dubberly Design Office · From Devices to Platforms · April 15, 2016
Now imagine Barbie’s AI connected to a CRM system.“Have you told Grandma what you’d like for your birthday?”
20Dubberly Design Office · From Devices to Platforms · April 15, 2016
What does the IoT mean for healthcareand healthcare usability?
21Dubberly Design Office · From Devices to Platforms · April 15, 2016
Consumer tracking devices (AKA “wearables”)have become common.
Apple Watch Basis B1 Fitbit Flex
Narrative Clip
Microsoft Band
Misfit Shine Nike Fuel
Jawbone Up
Moto 360
Garmin Vivofit
Pebble Classic Samsung Gear
Cloud
Accelerometer
Phone App
RAM
Analytics engine
Friends
Wi-Fi
Website
22Dubberly Design Office · From Devices to Platforms · April 15, 2016
Sensors collect data and pass it to web-based apps;users set goals and share data with friends and family.Feedback loops can promote change and help manage health.
23Dubberly Design Office · From Devices to Platforms · April 15, 2016
Advanced sensors will change the meaning of “wearables”.Google’s Project Jacquard is creating touch-sensitive fabric.
https://www.google.com/atap/project-jacquard/
HCP
Wi-Fi
Website
Sensor
RAM
Phone
PatientFamily
Cloud
Blood
App
Analytics engine
24Dubberly Design Office · From Devices to Platforms · April 15, 2016
Medical devices are connecting to the cloud, too,including pacemaker-defibrillators, autoinjectors, and glucometers.Smart, connected glucometers can alert family and HCPs about extreme lows;they also collect data automatically and help users see trends and make correlations.
Cartridge(disposable)
Plunger(disposable)
Cartridge Cap(disposable)
Needle(medical sharps,
disposable)
Needle Cap(disposable)
Insulin Vial(disposable)
Syringe Filled cartridgesyringe disassembled &cartridge cap connected.
Filled cartridgecasket cap threaded onto tubing &infusion kit connected to cartridge.(cartridge and tubing disposable)
Animas® Inset®
(medical sharps,disposable)
ReservoirPackaging
(disposable)
InsulinPackaging
(disposable)
Animas® Inset®
Packaging(disposable)
Reservoir Kit(disposable)
Inset Kit(disposable)
contains 10single use
contains
containscontains 10single use
use inserterto apply
consists of creates load into
fill thesyringe
attachinfusionkit tubingto cartridge
attachinfusion
kit tubing
patientrecieves
patientrecieves
patientrecieves
flexibleplastictube
OneTouch® Ping®
Insulin Pump
Infusion Set(disposable)
25Dubberly Design Office · From Devices to Platforms · April 15, 2016
And they are connecting to other smart, connected devices.A smart glucometer + a smart insulin pump = artificial pancreas
Endocrinologists may be able to set doses remotely.Systems could enable direct contact; e.g., “Do you feel nausea?”
Remote monitoring+ intervention DB
EMR
Nurse
Wi-Fi
Website
Pumps
Phone
Patient
On-board drug library
IV bags + lines
Cloud
Analytics engine
26Dubberly Design Office · From Devices to Platforms · April 15, 2016
Similar changes are beginning in clinical settings, too.Large volume infusion pumps (LVPs) used to stand-alone.
Nurse
Wi-Fi
Cloud
Air mattresssensors
Patient
Website
Phone App
27Dubberly Design Office · From Devices to Platforms · April 15, 2016
Even hospital beds will become smart, connected devicesand join the medical product-services ecology.Already, some beds can measure:
– Patient weight – Heart rate – Breathing rate – Intensity and duration of movement – Bed entrance + exit – Sleep
http://smartbed.goodmarkmedical.com/smart-bed-technology-new/learn-smart-bed-solution/
Customer database
BatteriesMicroprocessor
Cloud
SpeakerAImachine-learningsystem
Microphone
Wi-Fi
28Dubberly Design Office · From Devices to Platforms · April 15, 2016
Catalia Health is developing Mabu, a personal healthcare companion.Robot + touch-screen + network + cloud services = Mabu
– Recognizes you and what you say – Learns about you – Imports data from health trackers – Converses with you about your condition – Reminds you to take medications – Connects with HCPs
29Dubberly Design Office · From Devices to Platforms · April 15, 2016
10 11 12PM
1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11 12PM
1 2 3 4 5 6 7 8 9
HEART R ATE (BEATS PER MINUTE)
ACCELER ATION (G FORCE)
ELECTRODERMAL ACTIVIT Y (MICRO-SIEMENS) Gray = OriginalBlue = Processed
0
5
10
15
20
25
30
0
20
40
60
80
100
120
140
160
180
200
220
0
0.25
0.50
0.75
1.00
1.25
9Day 1
12Day 2
IoT devices will generate huge amounts of data,requiring monitoring, analysis, and visualization.Stress data from a RWJF study of family caregiving,over ~24 hours, 20 participants with a dozen sensors, generated 5GB of data.
Gra
y =
Ori
gina
lB
lue
= Pr
oces
sed
9D
ay 1
1011
12 PM1
23
45
67
89
1011
12D
ay 2
12
34
56
78
910
1112 PM
12
34
56
78
9
Front doorFay’sbedroom
Josephine’sbedroom
BathroomLiving roomB
KitchenLiving roomA9
Day
110
1112 PM
12
34
56
78
910
1112
Day
21
23
45
67
89
1011
12 PM1
23
45
67
89
9D
ay 1
1011
12 PM1
23
45
67
89
1011
12D
ay 2
12
34
56
78
910
1112 PM
12
34
56
78
9
9D
ay 1
1011
12 PM1
23
45
67
89
1011
12D
ay 2
12
34
56
78
910
1112 PM
12
34
56
78
9
30 20 10 0405060708090100
NO
ISE
(DEC
IBEL
S)
CO
2 (P
AR
TS
PER
MIL
LIO
N)
HU
MID
ITY
(P
ERC
ENT
)
30 20 10 0405060708090100
MO
TIO
N
9D
ay 1
1011
12 PM1
23
45
67
89
1011
12D
ay 2
12
34
56
78
910
1112 PM
12
34
56
78
9
AC
TIV
ITIE
S
9D
ay 1
1011
12 PM1
23
45
67
89
1011
12D
ay 2
12
34
56
78
910
1112 PM
12
34
56
78
9
9D
ay 1
1011
12 PM1
23
45
67
89
1011
12D
ay 2
12
34
56
78
910
1112 PM
12
34
56
78
9
9D
ay 1
1011
12 PM1
23
45
67
89
1011
12D
ay 2
12
34
56
78
910
1112 PM
12
34
56
78
9
HE
AR
T R
AT
E (B
EA
TS
PER
MIN
UT
E)
AC
CEL
ERA
TIO
N (
G F
OR
CE
)
ELEC
TR
OD
ERM
AL
AC
TIV
ITY
(M
ICR
O-S
IEM
ENS
)
051015202530 020406080100
120
140
160
180
200
2200
0.25
0.50
0.75
1.00
1.25
Prim
ary
unit
in li
ving
roo
m B
Seco
ndar
y un
itin
bat
hroo
m
Prim
ary
unit
in li
ving
roo
m B
100 0
200
300
400
500
600
700
800
900
1,00
01,
100
1,20
01,
300
1,40
01,
500
Care
givi
ngSt
ress
leve
l 5St
ress
leve
l 4St
ress
leve
l 3St
ress
leve
l 2St
ress
leve
l 1St
ress
leve
l 0
Self
Car
e
Leis
ure
Wor
k
Oth
er
Slee
p
Closing the windows before leaving the house causes the CO2 to increase, and noise to decrease.
Spike in humidity matches self-reported shower.
Motion detectors show that Fay’s mother Josephine went to the bathroom in the middle of the night.
Spike in acceleration may be driving to the grocery store. Spike in EDA may indicate driving was stressful.
Spikes in acceleration, EDA, and heart rate match self-reported shower.
Sel
f-re
po
rted
sle
epS
elf-
rep
ort
ed g
roce
ry s
ho
pp
ing
30Dubberly Design Office · From Devices to Platforms · April 15, 2016
Comparing data sets enables us to recognize patternsand becomes the basis for ongoing machine learning.
1. Product
Glucometer + processor = computer that can run apps; e.g., bolus calculator, calorie estimator and tracker.
2. Smart product
Software
3. Smart, connected product
Remotedevices
Software
When a glucometer connects to a smart-phone, cost can come down,because the meter can build on the phone's processor and display.Plus data can be shared with family and HCPs.
4. Product system
Monitor
Syringe
Cartridge
Vial
Pump
GlucoseManagement
System
A near continuous glucose monitor can be coupled with an insulin pump,forming a glucose management system.
Remotedevices
Software
5. Systems of systems
The glucose management system can connect with many other systems,such as EMRs, remote alerting, patient population management,drug interaction monitoring, pharmacy order management, and billing.
Monitor
Syringe
Cartridge
Vial
Pump
GlucoseManagement
System
HealthManagement
System
PharmacyOrder
ManagementSystem
EMRSystem
RemoteAlertingSystem
DrugInteractionMonitoring
SystemBillingSystem
PatientPopulation
ManagementSystem
Remotedevices
Software
31Dubberly Design Office · From Devices to Platforms · April 15, 2016
In the future, medical products will no longer stand alone.Increasingly, they will exist in complex service webs.
32Dubberly Design Office · From Devices to Platforms · April 15, 2016
As medical products connect with service webs,they are also connecting with drug development systems.
– Systems for funding research and development, creating and protecting “intellectual” property, and rewarding investment
– The drug-knowledge-package system
– Compound sourcing, manufacturing, and distribution systems
– Drug delivery devices
– Systems for educating physicians and patients
– Systems for helping patients integrate the drug into their lives
– Insurance and government payment systems
– Government regulatory systems and professional association practices
Patient Web + mobile
Bio-med engineers
Researchers
Designers
HCPs
Individual view
Individual view
Aggregate view
Individual view
Devices
Cloud
Integrateddatabases
Company views
Patient view- My Record- Messages- Appointments- Billing- Support
Patient ID#
Patient ID#
Patient ID#
All Patient Data
My Record
33Dubberly Design Office · From Devices to Platforms · April 15, 2016
Unified patient and device data will afford useful views to many constituents.
Integrateddatabases
Company views
Patient ID#
Patient ID#
Patient ID#
All Patient Data
Patient
Bio-med engineers
Researchers
Designers
HCPs
Individual view
Individual view
Aggregate view
Individual view
Devices
Cloud
Benefit = better experience
Web + mobile Patient view- My Record- Messages- Appointments- Billing- Support
- Clearer picture of health status- Alerts on possible problems- Personalized communications- Record of appointments
My Record
34Dubberly Design Office · From Devices to Platforms · April 15, 2016
Patients can know much more about what’s happeningand can share information with family, friends, and HCPs.
Integrateddatabases
Company views
Patient ID#
Patient ID#
Patient ID#
All Patient Data
Patient
Bio-med engineers
Researchers
Designers
HCPs
Benefit = higher patient sat
Individual view
Individual view
Aggregate view
Individual view
Devices
Cloud
Web + mobile Patient view- My Record- Messages- Appointments- Billing- Support
My Record
35Dubberly Design Office · From Devices to Platforms · April 15, 2016
HCPs can receive a more holistic view of each patientand can manage groups of patients more efficiently.
Patient
Bio-med engineers
Researchers
Designers
HCPs
Benefit = lower costs
Individual view
Individual view
Aggregate view
Individual view
Devices
Cloud
Web + mobile Integrateddatabases
Company views
Patient view- My Record- Messages- Appointments- Billing- Support
Patient ID#
Patient ID#
Patient ID#
All Patient Data
My Record
36Dubberly Design Office · From Devices to Platforms · April 15, 2016
Bio-med engineers can better manage equipment,improve service, and reduce support costs.
Patient
Bio-med engineers
Researchers
Designers
HCPs
Benefit = more efficient ops
Individual view
Individual view
Aggregate view
Individual view
Devices
Cloud
Web + mobile Integrateddatabases
Company views
Patient view- My Record- Messages- Appointments- Billing- Support
Patient ID#
Patient ID#
Patient ID#
All Patient Data
My Record
37Dubberly Design Office · From Devices to Platforms · April 15, 2016
Researchers can learn from aggregate data,to improve procedures and care-facility operations.
Patient
Bio-med engineers
Researchers
Designers
HCPs
Benefit = improved next gen products
Individual view
Individual view
Aggregate view
Individual view
Devices
Cloud
Web + mobile Integrateddatabases
Company views
Patient view- My Record- Messages- Appointments- Billing- Support
Patient ID#
Patient ID#
Patient ID#
All Patient Data
My Record
38Dubberly Design Office · From Devices to Platforms · April 15, 2016
Designers can get detailed usage data,to improve next generation products.
39Dubberly Design Office · From Devices to Platforms · April 15, 2016
Questions raised by the Internet of Things:
Of course, more UIs will have to be validated, but will FDA require the whole interconnected system to be validated together or treat each component as a separate device?
Will all parts of the system be considered Class 2 or 3, or will parts of it be Class 1?
How will usability engineering and usability testing change?
What other skills will usability professionals need?
What changes in methods will be required?
40Dubberly Design Office · From Devices to Platforms · April 15, 2016
The IoT signals a larger cultural shiftFrom
Industrial Age
Cathedral
Age of Enlightenment
Machine metaphors
Tree structures (e.g., the tree of life)
Direct causality
Patient coordination of specialists
“Mechanism of disease”
To
Information Age
Bazaar
“Age of Entanglement”
Ecology metaphors
Network structures(e.g., web of life)
Systemic causality
“Patient medical home” or “patient population management”
“Context of living in which disruptions arise” or other “integrative approaches”
—Eric Raymond
—Danny Hillis
—Manual Lima
—George Lakoff
—Fritjof Capra
[email protected]@dscience.com
Presentation posted athttp://presentations.dubberly.com/HFES_health.pdf
Special thanks to Lynn StrotherAnthony AndreSean HägenRyan Reposar