national association of regulatory commissioners – february 2008 page 1 of 13 national association...
TRANSCRIPT
National Association of Regulatory Commissioners – February 2008 Page 1 of 13
National Association of Regulatory Utility Commissioners
Winter Committee Meeting_________________________________________________________
Energy Efficiency Program Performance
Evaluation Issues from the States
Nick Hall
TecMarket Works
National Association of Regulatory Commissioners – February 2008 Page 2 of 13
5 Topics Covered Today
1. Changing Importance of Evaluation– Need for accuracy – reliability - objectivity
2. Reliability & Budgeting for Evaluation
3. Evaluation Approaches Used
4. Net-to-Gross Issues– Definitions change from state to state– Net savings are difficult to assess– Approaches for crediting savings
5. Changing way EE is viewed / assessed
National Association of Regulatory Commissioners – February 2008 Page 3 of 13
1. Changing Importance of Evaluation
Evaluation provides decision-support information in a changing environment…
–More people understand the true cost of fossil fuel & nuclear power supply and are demanding change. –Energy Efficiency represents the single largest impact potential for greenhouse gas reduction.–Energy efficiency is moving to be the resource of choice.–Public resistance to new plants, including some renewable energy facilities.–ISO’s, Commissioners, ALJs and others are asking for more and more reliable evaluations.
Accurate information is essential for Policy Decisions.
National Association of Regulatory Commissioners – February 2008 Page 4 of 13
2. Reliability & Evaluation Budgeting
Several state are worried about the accuracy and reliability of evaluation results
– Accuracy & reliability are controlled by the evaluation budget & the evaluation timeline.
– States are struggling with evaluation budget decisions.
– Some are legislating budgets without understanding the effects.
– Currently some states are setting evaluation budgets below the level of accuracy desired.
National Association of Regulatory Commissioners – February 2008 Page 5 of 13
2. Reliability & Evaluation Budgeting
Relationship between funding and results
Strong Funding – Right Approach
Weak Funding – Right Approach
Weak Funding – Wrong Approach
Strong Funding – Wrong Approach
National Association of Regulatory Commissioners – February 2008 Page 6 of 13
2. Budgeting for Evaluation
NY 2% Moved from 1%ILL 3% Set in legislationWI 4% Moved up from 3%MI 3-5% Proposed - Now being discussedKY 5% Proposed - Now being discussedOH 5% Proposed - Now being discussedNC 5% Proposed - Now being discussedSC 5% Proposed - Now being discussedIN 5% Proposed - Now being discussedCA 8% Moved up from 4.25%
National Association of Regulatory Commissioners – February 2008 Page 7 of 13
3. Evaluation Approaches Used
Impact Evaluation – Energy ImpactsMajority of Studies Fall Under 5 General Approaches*.
States pick and choose according to budget and reliability needs.
1.Engineering Estimates
2.Statistically adjusted Engineering Estimates
3.Building Meter/Billing Analysis
4.Measure Level Metering & Monitoring
5.Building Modeling and Simulations * See California Evaluation Framework/Protocols for others.
National Association of Regulatory Commissioners – February 2008 Page 8 of 13
3. Evaluation Approaches Used
Impact Evaluation – Energy ImpactsReliability Issues
1.Identifying and setting the baselines
2.On-site confirmation / metering / assessments
3.Sample Sizes
4.Representativeness of the Sample
5.Type of metering or monitoring used
6.Accuracy of self-reports
National Association of Regulatory Commissioners – February 2008 Page 9 of 13
4. Net to Gross Issues
Gross savings tend to be defined the same way.
Ex Ante - Gross
• Ex ante savings x tracking system measure installs.
• Ex ante savings x measure installs x verification rate.
Ex Post - Gross
• Ex post savings x measure installs x verification rate.
Net savings tend to be defined in different ways.• Net Savings = Gross Savings +/- Net Adjustments
National Association of Regulatory Commissioners – February 2008 Page 10 of 13
4. Net to Gross Issues
Net savings definitions and measurement approaches are not consistent across states, and sometimes across studies within a state.
Different definitions from state to state• Freeriders (10% to 60% of the program-claimed savings)
• Participant spillover (5% to 40% of the program-claimed saving)
• Short term non-participant spillover - market effects (5% to 20% of the program-claimed savings)
• Long-term non-participant spillover market effects (40% to >500% of the program-claimed savings)
Study results are inconsistent and not comparable.
National Association of Regulatory Commissioners – February 2008 Page 11 of 13
4. NTG Definitions Used
NY Gross – freeriders Can adjust for PS & MEILL Gross – freeriders + part spillover Not yet agreedWI Gross – freeriders + ME As documented in evalMI Not yet determined Not yet agreedKY Gross – freeriders + adjustments As documented in evalOH Gross – freeriders + adjustments As documented in evalNC Gross – freeriders + adjustments As documented in evalSC Gross – freeriders + adjustments As documented in evalIN Gross – freeriders + adjustments As documented in evalCA Gross – freeriders Now looking at options
National Association of Regulatory Commissioners – February 2008 Page 12 of 13
5. Changing way EE is viewed / assessed
State are considering changes that will impact EE evaluation and cost effectiveness assessments.
1. Elimination of policy-set life-cycle periods (EUL)
2. Valuing energy saved at market prices
3.Valuing greenhouse gas at total societal costs, not traded values
4.Setting renewable energy as the cost alternative to EE in cost comparison approaches (i.e. avoided costs = RE)
5. Discontinuing discounting of future savings.
6. EE as a profitable energy supply choice.
National Association of Regulatory Commissioners – February 2008 Page 13 of 13
Thank You
Nick Hall
TecMarket Works
165 West Netherwood Road
Suite A, 2nd Floor
Oregon, Wisconsin 53575
608 835 8855