exploring deep savings: a toolkit for assessing behavior-based energy interventions

Diffusion of Feedback Perceptions and Adoption of Devices

Exploring Deep Savings:A Toolkit for Assessing Behavior-Based Energy Interventions

Beth KarlinTransformational Media Lab

University of California, Irvine

Papers listed here. Contact me for details: [email protected].

Is Feedback Effective?100+ studies conducted since 1976Total n = 256,536 (mean 119/study)Mean r-effect size = .1174 (p < .001)Average energy savings: 9%

Significant variability in effects (from negative effects to over 20% savings)Karlin, Ford & Zinger. (In Press). The Effects of Feedback on Energy Conservation: A Preliminary Theory and Meta-Analysis. Psychological Bulletin.

When we have no idea we might unplug to be safe. When we know 2

Is Feedback Effective?It dependsModerators identified in meta-analysisStudy population (WHO?)Study duration (HOW LONG?)Frequency of feedback (HOW OFTEN?)Feedback medium (WHAT TYPE?)Disaggregation (WHAT LEVEL?)Comparison (WHAT MESSAGE?)

Karlin, Ford & Zinger. (In Press). The Effects of Feedback on Energy Conservation: A Preliminary Theory and Meta-Analysis. Psychological Bulletin.

When we have no idea we might unplug to be safe. When we know 3

Methodological LimitationsNot naturalisticParticipants generally recruited to participateMay be different from active adoptersNot comparativeMost studies tests one type of feedback (vs. control)Very few studies isolating or combining variablesNot testing mediationDV is energy use, but studies rarely test possible mediators to explain effectiveness


Infrastructure is coming with the smart grid, technology is developed and being fine tuned via multiple presentation mediums AND culturally, were already there. Feedback is everywhere, calorie count of food, cotton counts on clothing. Information is popular!4

Methodological LimitationsNot naturalisticParticipants generally recruited to participateMay be different from active adoptersNot comparativeMost studies tests one type of feedback (vs. control)Very few studies isolating or combining variablesNot testing mediationDV is energy use, but studies rarely test possible mediators to explain effectiveness



Will they come?

If you build it,

Infrastructure is coming with the smart grid, technology is developed and being fine tuned via multiple presentation mediums AND culturally, were already there. Feedback is everywhere, calorie count of food, cotton counts on clothing. Information is popular!

Program x

Outcome y

Does it work?Does program x lead to outcome y?


Program x

Outcome y

Questions remainWhat is the program?What is going on here?How do we measure outcomes?Does program x lead to outcome y?


Program x

Outcome y

A theoretical approachHypothesis / TheoryClearly defined and operationalizedMetrics tested for reliability & validityDoes program x lead to outcome y? How and for whom does program x lead to outcome y?


How and For Whom?

B. Karlin

10

How and for Whom?

11

Toolkits in other fields

65.9 billion kWh5.8% of electricity in average home7th most consuming activity behind: air conditioning, refrigeration, space heating, water heating, and lighting.Cost: 30-40 cents in an electric dryer; 15-20 cents in a gas dryer. Over its expected lifetime of 18 years, the average clothes dryer will cost you approximately $1,530 to operate.

Do Not Use a Clothes Dryer........................ 23.6 million (21.2%)Use a Clothes Dryer.................................... 87.5 million (78.8%)12

Toolkits in other fields



Question Bias



Question Bias



Question Bias



Psychometrics

Theory and technique of measurement: knowledge, abilities, attitudes, traitsConstruction and validation of instruments: questionnaires, tests, assessments

17

Psychometrics

Psychometric Properties Factor Structure Reliability Criterion Validity Sensitivity

18

Our Project

19

Methodological Review

Karlin, Ford, Wu, & Nasser. (2015). What Do We Know About What We Know? A Review of Behaviour-Based Energy Efficiency Data Collection. IEA-DSM Task 24 Subtask 3 Report.

Karlin, Ford, Wu, & Nasser. (2015). What Do We Know About What We Know? A Review of Behaviour-Based Energy Efficiency Data Collection. IEA-DSM Task 24 Subtask 3 Report. Data Collection Methods Used

Instruments Provided?

Karlin, Ford, Wu, & Nasser. (2015). What Do We Know About What We Know? A Review of Behaviour-Based Energy Efficiency Data Collection. IEA-DSM Task 24 Subtask 3 Report.

Toolkit Development

Energy Cultures Frameworks

Material cultureEnergy practicesNorms and aspirationsHaveDoThink

Toolkit DevelopmentHeating devicesEnergy sourcesInsulationHouse structure

Have

Toolkit DevelopmentSocial expectations and aspirationsEnvironmental concernExpected warmth levelsMaintaining traditions

Think

Toolkit DevelopmentTurning on heaterPutting on jerseyMaintaining heating technologiesDrawing curtains

Do

Toolkit Development

28

Toolkit Development

29

Next StepsPsychometric testing (SCE)Local field testing (PG&E / SCE)Member country reviewGlobal field testingWide scale adoption?


Beth KarlinTransformational Media Lab

University of California, [email protected] you!(comments and suggestions welcome)

Material Culture (Have)

1. What type of dwelling do you live in?

2. Which of the following appliances do you own? (Space heater, dishwasher, central AC, etc.)

Beliefs (Think)

Environmental Concern

I consider myself to be an environmentalist

Norms (Personal and Social)

1. I feel a strong personal obligation to conserve energy.

2. Most people are not willing to make changes or sacrifices to protect the environment.

Efficacy (Performance and Response)

3. I can think of at least one thing that I can do to decrease my energy usage.

4. If I conserve, it will have a positive societal impact.

Motivation

5. How much does each of the following factors affect your household energy use?

(Environmental impact, cost of energy bill, convenience, etc.)

Behavioral Intention

6. During the next six months, I intend to (limit time in shower, turn of lights when not needed, etc.)

Behaviors (Do)

1. How frequently do you: (Limit time in shower, turn off lights when not needed, etc.)

User Experience (UPscale)

Ease of Use

1. I feel very confident interpreting the information provided to me.

2. A person would need to learn a lot in order to understand this _______________.

Engagement

3. I do not find this ____________ to be useful.

4. I think that I would like to use this _______________ frequently.