2020-2021 impact evaluation plan - bpa.gov · evaluation prior to start and provided with clear...
TRANSCRIPT
![Page 1: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/1.jpg)
2020-2021 Impact Evaluation Plan
Brown BagJanuary 2020
1
![Page 2: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/2.jpg)
Agenda
Evaluation Planning Process
Why Evaluation
2Questions?
FY20-21 Plan Draft OverviewCustom and C/I/Ag Lighting
Other areas
![Page 3: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/3.jpg)
3
BPA Core Team
Planning and Evaluation
Carrie Nelson
Phillip Kelsven
EER/Marketing
Melissa Podeszwa
Michele Francisco
Engineering
Keshmira McVey
![Page 4: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/4.jpg)
Contractor Team
4
Steve GroverProject Director
Tami RasmussenProject Manager
Mike BakerEngineering Lead
Lauren GageStakeholder Management
![Page 5: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/5.jpg)
Why Evaluation?
5
![Page 6: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/6.jpg)
What do we all want?
6
Energy efficiency programs that save customers money and energy.
To be trustworthy stewards of their money.
![Page 7: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/7.jpg)
7
How do we improve?
What did we achieve?
Evaluation
![Page 8: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/8.jpg)
8
![Page 9: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/9.jpg)
• Identifies implementation requirements and includes requirement to provide access to project, documentation and billing data for evaluation and oversight purposes.
Implementation Manual
• Defines BPA’s impact evaluation activities as minimum of 80% of portfolio every 4 years. Consistent with RTF Guidelines and national standards.
• BPA M&V Protocols.
BPA Internal Policies
9
Relevant Evaluation Policies
![Page 10: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/10.jpg)
Evaluation Planning Process
10
![Page 11: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/11.jpg)
Evaluation Planning Considerations
Evaluation should focus on areas that are not covered by previous evaluation or other
research activities (e.g., engineering, NEEA, RTF)
Areas of savings uncertainty benefit from evaluation learnings
To meet the internal BPA policy goals and focus on largest portfolio impacts, savings should be large
Future areas with increasing savings add insight to BPA and the region
Evaluation should support programs, engineering,
momentum savings wherever possible
11
![Page 12: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/12.jpg)
Assess & understand portfolioLinked: BOOM data, UES Measure List and RTF measure status,
historic evaluation coverage, engineering priorities, research activities for NEEA, RTF, Momentum Savings
Identify potential evaluation opportunities
Draft Evaluation Plan
Finalize Plan: Review, discuss, reviseInternal core team and program teams
Brown bag
Evaluation Planning Process
12
![Page 13: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/13.jpg)
2018 Portfolio Savings:By Sector
Commercial dominated 2018
savings
13
![Page 14: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/14.jpg)
2018 Portfolio Savings: By Measure Type
Custom projects, lighting calculators, SEM are 70% of portfolio
14
![Page 15: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/15.jpg)
Evaluation Coverage
From 2012-19, BPA evaluated 89%* of portfolio
Most coverage has expired (i.e., more than 4 years old)
– ~7% covered by current UES billing analysis
15* Based on 2018 savings.
![Page 16: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/16.jpg)
FY2020-2021 Evaluation Plan
16
![Page 17: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/17.jpg)
1717
Overview of FY20-21 Evaluation Areas
Custom and C/I/Ag Lighting
Engineering-based evaluation of projects
Strategic Energy Management
Persistence assessment to inform measure life and (if feasible) assessment of how capital
measures affect SEM savings
BPAQ HVAC Commercial
Billing analysis to support BPAQ measure assessment
17
![Page 18: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/18.jpg)
1818
Project Documents
Electric bills/AMI
Customer Surveys
Customer site visits
Delivery Verification
Billing Analysis
Engineering Modeling
DATA COLLECTION
SAVINGS ESTIMATION
More informationMore effort
Less informationLess effort
![Page 19: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/19.jpg)
1919
Evaluation Activities by Measure Group
*Activity only if needed
Data Collection ActivitiesSaving Estimation
Methods
Project Documents
Billing Data
Customer Surveys
Site Visits
Engineering Modeling
Billing Analysis
Custom and C/I/Ag Lighting
*
Strategic Energy Management
Persistence *
Capital *
BPAQ HVAC Commercial *
19
![Page 20: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/20.jpg)
2020
Evaluation Activities by Measure Group
AreaFY2020- 21 Sample Size
TechniqueSampled
Years
Custom and C/I/Ag Lighting ~113
Savings stratified random sample
One-year prior for each domain
Strategic Energy
Management
Persistence ~15 Simple random 2015-2019
Capital ~10 Simple random One year prior
BPAQ Commercial HVAC ~150 Census 2017-2019
20
![Page 21: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/21.jpg)
2121
Evaluation Activities: Timing
21
FY2020 FY2021
Evaluation Activity Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Custom and C/I/Ag Lighting
Option 1 Custom Industrial
Option 2 Custom IndustrialOption 1 LightingOption 2 Lighting
SEM Evaluation
BPAQ Commercial HVAC
Clark Thermostat Pilot Evaluation
(FY2019) Residential HVAC Evaluation
Planning Data Collection/Analysis
Draft Report and Review
Report and Results Communications
![Page 22: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/22.jpg)
Utility and End-User Contact Principles
BPA provides opportunities for utilities to understand the details of the
evaluation plan and data request.
Utilities are notified of projects in evaluation prior to start and provided
with clear information on samples, timelines, and requirements.
BPA provides minimum of 2 weeks notice to utilities prior to end-user contact, including phone surveys
and site visits.
Utilities have reasonable timeline to collect project and billing data;
escalation protocol of timelines missed.
22 22
![Page 23: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/23.jpg)
Evaluation and Oversight Coordination
Share focus areas, processes, and results
Goals: minimize impacts on customers, maximize information sharing
23
We are working on the ability to cross-reference sampled projects with COTR-collected project
information to minimize customer effort.
![Page 24: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/24.jpg)
Custom and C/I/Ag Lighting
24
![Page 25: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/25.jpg)
Staged evaluation• Focus areas (domains)• Smaller sample sizes• Bi-annual sampling• Distributed over 4 years
Leverage inspection reports/QC
by utility and BPA to avoid site-visits wherever possible
Custom and C/I/AgEvaluation ideas
25
![Page 26: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/26.jpg)
Custom and C/I/Ag Lighting Staged Timing
26 26
FY20 FY21 FY22 FY23
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Option 1 Utilities
Custom Industrial
Lighting C/I/Ag
Custom Non-IndustrialOption 2 Utilities
Custom Industrial
Lighting C/I/Ag
Custom Non-Industrial
ESRP Federal
Data Collection/Analysis
Draft Report and Review
Final Report and Results Communications
![Page 27: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/27.jpg)
Custom and C/I/Ag General Approach
27
Reporting and recommendations
Contactsite
Analyze savings
Sample sites
If needed, site visit Review
documentation
![Page 28: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/28.jpg)
Other Evaluation Areas
28
![Page 29: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/29.jpg)
Strategic Energy Management
• Determine how long SEM-initiated changes are operational
Investigate Persistence of SEM Changes
• Verify capital measure savings, determine if errors/ incompatibilities create errors in SEM savings
Verify Capital Savings Deduction
Historic evaluation showed reliable MT&R modeling. Recommend new objectives:
29 29
![Page 30: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/30.jpg)
BPA-Qualified Commercial HVAC
• DHP have most units: 124 in 2018
BPA offers ASHP and DHP measures as BPA Qualified
• Will collect more for recent projects
Evaluation collected project data and billing data on ~105 projects in 2019
• Pooled billing analysis• Survey – if needed
Approach
30 30
![Page 31: 2020-2021 Impact Evaluation Plan - BPA.gov · evaluation prior to start and provided with clear information on samples, timelines, and requirements. BPA provides minimum of 2 weeks](https://reader033.vdocuments.site/reader033/viewer/2022050503/5f95998ee6f481007341c9f2/html5/thumbnails/31.jpg)
• Commercial VFDs• Commissioning/Controls/
Sizing efficient case research (if needed after baseline research)
• Other new measures (e.g,. New BPAQ measures)
• Low rise multifamily• Understanding building shell
for residential HVAC
• Commercial thermostats • C&I Lighting controls• Showerheads• Showerheads/TSV• Power strips (res and com)• Ag: Hardware, pumps• Low-E storm windows• Commercial pumps and fans
Evaluation Watch List
31 31