quantifying usability programmable...
TRANSCRIPT
How People Actually Use Thermostats
Quantifying Usability in Programmable Thermostats
Alan Meier* and Cecilia Aragon – LBNL
Therese Peffer and Daniel Perry – UC Berkeley
Marco Pritoni – UC Davis
Thermostats
• In 2008 Energy Star concluded that homes with programmable thermostats were using more energy than homes with manual thermostats.
• Energy Star terminated the program and began developing a new specification
Premise: Improving the usability of thermostats will facilitate energy‐saving behavior
This Talk
1. Surveys of usability of thermostats in homes2. Methodology for quantifying usability of
thermostats and other controls
Conclusions:• Few homes exploit full potential of programmable thermostats
• It is possible to quantify usability in a way suitable for use in design specifications for standards
Are Programmable Thermostats Used Correctly (or features fully exploited)?
Parallel investigations via:
1. Weatherization crews
2. Amazon Mechanical Turk
3. Interviews and other on‐line surveys
-“Do you use the programming features of the thermostat?”
- “Yes”
-“Are you satisfied with your thermostat?”
- “It’s OK”
-“If you could have a new thermostat, what would you like it to do differently from your current thermostat? (language, font size, button size, technical terms)?”
- “NO”
“The thermostat is on Hold”
Survey of a Weatherized Home
Weatherization Survey Results
• 20 homes visited (in mid‐west)
• 45% on “hold”• 5% switched off (in winter)
Talk to us if your wxncrew would like to participate in Round 2 of this survey
Amazon Mechanical Turk Survey
The Workers:
1. Fill out surveys on the web
1. Photograph their thermostats
1. Get paid via Amazon
We get rapid, tabulated, results!
Amazon Mechanical Turk Results
• 63 responses in 24 hr
• ~20% had errors in time setting
• ~50% on long‐term hold
• Next survey will collect several hundred responses
Examples of uploaded photos
And Now the Video…
How to Measure a Product’s Usability?
1. Define tasks
2. Quantify peoples’ ability to accomplish tasks
3. Compute “score” based on metrics
4. Compare to reference model
Details of Usability Tests
• 5 thermostat interfaces• 31 participants• 2 interfaces per person• 6 tasks for each test• 372 videos
Thermostats Tested
Define Tasks
Task 1: Turn the thermostat from “off” to “heat.”Task 2: Set the correct time.
Task 3: Identify the temperature the device is set to reach.
Task 4: Identify what temperature the thermostat is set to reach for Thursday at 9:00 PM.
Task 5: Put the thermostat in “hold” or “vacation” to keep the same temperature while gone.
Task 6: Program a schedule and temperature preferences for Monday through Friday.
Task 1: Turn the thermostat from “off”to “heat.”
Distribution of Times for Subjects to Complete Task 1
Task 1: Turn the thermostat from “off” to “heat.”
Average Time to Complete Task by Model
Task 1: Turn the thermostat from “off” to “heat.”Task 2: Set the correct time.Task 3: Identify the temperature the device is set to reach.
Time to Complete Tasks by Model
Consumer Reports Recommended the Model that Ranked Lowest in Usability
Conclusions
Current usage of thermostats:– ~50% of programmable thermostats set to long‐term hold
– ~20% do not have the correct time
Many homes fail to use features of programmable thermostats
– Do people believe that installing a PT will automaticallysave energy?
We created and demonstrated a methodology to quantify usability
– Applies to HP water heaters, TVs, lighting controls, etc.
Premise: Improving the usability of thermostats will facilitate energy‐saving behavior
Future Work
• Expand surveys
• “Road Test” methodology
• Next steps in measuring usability3. Compute “score” based on metrics4. Compare to reference model
• Apply methodology to lighting, TVs, other appliances
Talk to us if your wxncrew would like to participate in Round 2 of this survey
Alan Meier [email protected]
Controls for Heat Pump Water Heaters (HPWHs)
1tute, Inc. All rights reserved.
Controls
GERheem
A. O. Smith
An incorrect setting could increase electricity use 1500 kWh/yr
Codes for Hunter Thermostat
HTR Codeen = energy button tu/td = set temperature up OR down
ht = home today button h/r = hold/ return button
tu = temperature up p = program button
td = temperature down pd = program day button
ct = current temp display on touchscreen [0] blank touchscreen space
ti = display time on touchscreen f = filter button
sun = sun icon on touchscreen fo = fan on switch
he = heat switch fa = fan auto switch
d/t = day/time button fof = fan off
“Lostness”: A Metric for Quantifying Usability
Minimum Keystrokes to
Accomplish Task
Actual Keystrokes by
Subject
cl C
cu/cd sy
c d
sr
c
cl
Cd x 10
cu
Cd x 3
Cu x 8
.
.
.
(3 keystrokes) (28 keystrokes)
“lostness coefficient”= 28/3~ 9
Keystrokes transcribed from videos
Why Do People Keep Thermostats on Long‐Term Hold?
• They don’t– The survey captured a rare event
• They don’t know how to easily operate other modes (including leaving this mode)– Usability problems
• They think this mode is energy‐saving (since they installed an “energy‐saving” thermostat)
• They want this setting– Technical implications about optimal design of controls, levels of efficiency investment
Mechanical Turk – ScreenshotsSurvey General
Settings
Mechanical Turk – ScreenshotsResults: Web View