Net to GrossJeff Guild

November 15, 2016

Questions?

• For context, how have NTG values been determined historically for RCx?

• Is there anything in the ongoing industry conversation that speaks to changes or vision around setting NTG?

• What can be done through program design that minimizes free-ridership?

• PG&E uses a free-rider form, could there be anything data-driven and more sophisticated to establish naturally occurring savings?

2

Discussion Topics

• Terminology

• Roles in Program Evaluation

• Methods for Evaluation

• What we typically do (under CPUC)

• In Practice

• Examples in RCx

• Policy Trends

3

Gross Savings - Defined

• Changes in energy consumption that result directly from program-related actions taken by participants of an EE program, regardless of why they participated.1

• 8the savings in energy and demand seen by the participant at the meter. 2

• 8count the energy savings from installed energy efficiency measures irrespective of whether or not those savings are from free riders, i.e., those customers who would have installed the measure(s) even without the financial incentives offered under the program. Gross savings are adjusted by a net-to-gross ratio to produce net savings, that is, to remove the savings associated with free riders. 3

4

1 2014 The Uniform Methods Project http://www.nrel.gov/docs/fy14osti/62678.pdf2 2001 Standard Practice Manual

http://www.cpuc.ca.gov/WorkArea/DownloadAsset.aspx?id=77413 2013 Energy Efficiency Policy Manual R5

http://www.cpuc.ca.gov/uploadedFiles/CPUC_Public_Website/Content/Utilities_and_Industrie

s/Energy/Energy_Programs/Demand_Side_Management/EE_and_Energy_Savings_Assist/E

EPolicyManualV5forPDF%20(1).pdf

Net Savings - Defined

• Changes in energy use that are attributable to a particular EE program. These changes may implicitly or explicitly include the effects of free ridership, spillover, and induced market effects.1

• 8are gross savings minus those changes in energy use and demand that would have happened even in the absence of the program. 2

• 8The savings realized when free ridership is accounted for. The savings is calculated by multiplying the gross savings by the net to gross ratio. 3

5

1 2014 The Uniform Methods Project http://www.nrel.gov/docs/fy14osti/62678.pdf2 2001 Standard Practice Manual

http://www.cpuc.ca.gov/WorkArea/DownloadAsset.aspx?id=77413 2013 Energy Efficiency Policy Manual R5

http://www.cpuc.ca.gov/uploadedFiles/CPUC_Public_Website/Content/Utilities_and_Industrie

s/Energy/Energy_Programs/Demand_Side_Management/EE_and_Energy_Savings_Assist/E

EPolicyManualV5forPDF%20(1).pdf

What’s a Baseline?

• The baseline is the least efficient, non-regressive, code or regulation-compliant option specific to a particular facility and application that the customer technically, functionally and economically could have alternatively considered to deliver the post-retrofit level of production or service.

• Baseline definition should determine the least efficient approach that a participant reasonably could have taken.

• The technical baseline should be separated as much as possible from the concepts of free ridership, which characterizes what the participant would have done, as opposed to the least efficient choice he or she could have made.

6

2011 ACEEE Summer Study Establishing Baselines for Industrial Energy Efficiency Projects

http://aceee.org/files/proceedings/2011/data/papers/0085-000031.pdf

Gross Savings – Fun Facts

• Typically used in program goals and

program performance measurement

• Typically the basis for customer

incentives

• Subject to “realization rate” adjustment

in Impact Evaluation (and prior to

“getting netted”)

7

Net Savings – Fun Facts

Counts towards ESPI Scores (IOU “Bonus”)Counts towards cost effectivenessCurrent method for demand forecastingUnits that need to be doubled (SB350)Adjustment Factors include:

• Free Ridership

• Spillover

• Market Effects

8

Baseline – Fun Facts

• Simplifies NTG evaluation by removing partial free riders

• Reduces risk of “double-counting” or overlapping with “Code Advocacy” program savings claim, especially for Normal Replacement or Early Retirement

• Personal experience – Code is not well understood and not always the proper baseline

• Note: Use of a “common practice” baseline is not necessarily common practice

9

Methods for Net Savings

Determination

10

1 2014 The Uniform Methods Project http://www.nrel.gov/docs/fy14osti/62678.pdf

Free Ridership

• CA has typically used the Survey

Method for Evaluation – Example:

11

1 2014 The Uniform Methods Project http://www.nrel.gov/docs/fy14osti/62678.pdf

Free Ridership

Survey Method - Pros• Can provide useful information to

support process and impact evaluations (for example, source of awareness, satisfaction, and demographics)

• Flexible approach that allows the evaluator to tailor questions to the program design or implementation methods

• Can yield estimates of free ridership and spillover without the need for a nonparticipant control group

12

1 2014 The Uniform Methods Project http://www.nrel.gov/docs/fy14osti/62678.pdf

Free Ridership

Survey Method - Cons• Potential biases related to respondents’ giving “socially

desirable” answers

• Consumers’ inability to know what they would have done in a hypothetical alternative situation, especially in current program designs that use multiple methods to influence behavior

• The tendency of respondents to rationalize past choices

• Potential arbitrariness of scoring methods based on evaluator judgment that translate responses into free rider estimates

• Consumers may fail to recognize the influence the program may have had on other parties who influenced their decisions (for example, program may have influenced contractor practices, which in turn impacted the participant)

• Participant surveys capture only a subset of market effects

13

1 2014 The Uniform Methods Project http://www.nrel.gov/docs/fy14osti/62678.pdf

Free Rider Prevention

14

• Does it align with how decision makers actually make decisions?

• Is it credible?

RCx Impact Evaluation

15

1 2010 ACEEE Summer Study Results from a Comprehensive Impact Evaluation of the

2006-08 California RCx Portfolio http://aceee.org/files/proceedings/2010/data/papers/2034.pdf

kWh Realization Rate?

0.62

Reasons for reductions include:

• Measure no longer operational

• Discrepancies between program calculation assumptions and actual conditions

• Changes in building operation

• Partial implementation

RCx Impact Evaluation

16

1 2010 ACEEE Summer Study Results from a Comprehensive Impact Evaluation of the

2006-08 California RCx Portfolio http://aceee.org/files/proceedings/2010/data/papers/2034.pdf

kWh Net to Gross?

0.84

Reasons for (only a modest) adjustment

• Programs that cover all or part of the cost of the RCx study reduce the risk associated with an RCx project significantly and lead many organizations to proceed with the project. Incentives that cover the cost of the study received the highest mean rating for all program influences cited by respondents—even higher than incentives for implementing recommended measures.

• RCx programs also make projects possible by helping offset funding cutbacks, staffing shortages, and reductions in maintenance budgets, particularly in public institutions, but also in hard-hit private sectors such as office buildings and the hospitality industry.

• The most significant non-program influences in the decision to pursue RCx projects appear to be government or corporate policies that require or encourage implementation of energy efficiency or other “green” measures.

Policy Trends

17

2016-08-18 Rulemaking 13-11-005 Decision

http://docs.cpuc.ca.gov/PublishedDocs/Published/G000/M166/K210/166210225.PDF

More Policy Trends

18

2016-08-18 Rulemaking 13-11-005 Decision

http://docs.cpuc.ca.gov/PublishedDocs/Published/G000/M166/K210/166210225.PDF

• Existing Conditions Baselines will put

greater pressure on Net Goals

• Will reduce pressure on overreaching

Codes that outpace the market and

enforcement competence

Thanks!

19