electric / gas / water summary of final evaluation report prepared by: rafael friedmann, pg&e...

Electric / Gas / Water

Summary of Final Evaluation Report

Prepared by:

Rafael Friedmann, PG&E

Kris Bradley & Christie Torok, Quantum Consulting

2003 Statewide Nonresidential Audit Program Evaluation

Knowledge to Shape Your Future2

• Tracking System Assessment– Verify tracking system accomplishments versus goals and

evaluate tracking system content

• Small Company Impact and Satisfaction Assessment– Measure first year program impact and customer

satisfaction with program components

• Large Company Impact Assessment– Measure program-related energy efficiency actions taken

over the first two years following the Audit

• Best Practices Assessment– Provide an up-to-date, prospective recommendation for the

best approach to on-site audits for different sized customers.

Evaluation Objectives


• Statewide the Audit program delivered audit services to over 29,000 utility customers, with over ½ of the completed audits provided to customers defined as hard-to-reach (HTR)

Q4 Report Goals Q4 Report GoalsPG&E 11,865 6,500 9,059 3,500 SCE 8,533 7,100 3,963 2,840 SDG&E 5,063 5,000 2,621 800 SCG 4,283 4,000 475 400 Total 29,744 22,600 16,118 7,540

Total Participation Hard-to-Reach Participation

Program Targets and Accomplishments


Data Collection Summary

• Smaller Customer Surveys (Less Than 100 kW)– 259 Participant / 261 Nonparticipant– Customers with Less Than 100 kW – PY 2003 Participants– Phone, Mail, On-Site, CD-ROM, On-Line

• Larger Customer Surveys (More Than 100 kW)– 84 Participant / 86 Nonparticipant – Customers with More Than 100 kW– PY 2002 Participants– On-Site Only

• Program Manager and Implementation Staff Interviews


• 25% of small and very small nonparticipating customers are aware of the audit program, versus about 40% of medium/large customers

25% 25%

37%

42%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Very Small(N=129)

Small(N=132)

Medium(N=37)

Large(N=49)

Rates of Audit Program Awareness in the General Population


• Awareness of the Audit Program in the general population is driven by the IOUs, who account for 70 percent of overall awareness

34%

9%

84%

50%

24%

18%

12%

22%

7%

3%

3%

23%29%

10%

28%

38%

--2%--1%

--1%--2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Very Small and SmallParticipants

(N=259)

Very Small and SmallNonparticipants

(N=59)

Medium and LargeParticipants

(N=84)

Medium and LargeNonparticipants

(N=33)

Other

RespondentContacted Utility

Bill Inserts

Utility Brochure inMail

Contacted by UtilityRepresentative

*Medium and large participants are PY2002 and onsite; very small and small participants are PY2003 and include all delivery mechanisms

Utility Marketing Sources of Program Awareness in the General Population


• Satisfaction levels with the Audit program are reasonably high, but participants desire a more useful audit product (report)

65%

70%74%

46%

27%

19%15%

32%

5% --2%

16%

--2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Overall Audit Program Quality of Report Credibility ofRecommendations

Usefulness

Unsatisfied(1-3)

SomewhatSatisfied (4-7)

Very Satisfied(8-10)

N=259

Satisfaction with Audit Program Elements Among Small and Very Small Participants


(N=86)

(N=261)

(N=84)

(N=255)

(N=84)

(N=259)

1

2

3

4

5

6

7

Small and Very Small Medium and Large

Mean K

now

ledge R

ati

ng (

1-1

0 s

cale

)

Nonparticipants

Participants - Before Audit

Participants - After Audit


Self-Reported Knowledge of Energy Efficiency

• Participants show a measurable increase in energy efficiency knowledge resulting from the audit


• Audit participants purchase rebated equipment at much higher rates than nonparticipants

0%

10%

20%

30%

40%

50%

60%

Small/Very Small Participant(N=259)

Small/Very SmallNonparticipant (N=261)

Medium/Large Participant(N=84)

Medium/LargeNonparticipant (N=86)

SPC

Express Efficiency

Unspecified Rebate Program

Rebate Program Participation Levels


• Participants want more customization in the audit reports (and recommendations), and want measure costs and rebates for use in ROI calculations

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Very Small and Small Participants(N=85)

Medium and Large Participants(N=15)

Less information

More communication &information

More personal contact

Provide results faster

Email info on rebates, EE

Give costs to replaceequipment

More customization


Participant Suggestions for Program Improvement


32%

24%

9%

20%

17%

12%

9%

12%

0%

5%

10%

15%

20%

25%

30%

35%

Lighting Cooling Gas Other

Adopti

on R

ate

Participant (N=259)

Nonparticipant (N=261)

Small and Very Small Customer Equipment Adoption Rates

• Among smaller customers, participant adoption rates for lighting and cooling, as well as ‘other’ miscellaneous equipment, are considerably higher than for a nonparticipant control group


N=51

N=33

N=84

N=84N=84

N=39

N=47 N=86

N=86N=86

0%

10%

20%

30%

40%

50%

60%

Lighting Cooling Gas Other Industrial Process

Adopti

on R

ate

ParticipantNonparticipant

Medium and Large Customer Equipment Adoption Rates

• Among larger customers, when comparing participants and a nonparticipant control group, marked differences in adoption rates exist for industrial process, lighting, cooling, and gas equipment


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%Very

Sm

all

an

d S

mall

(N=

66

)

Med

ium

an

dLa

rge

(N=

31

)

Very

Sm

all

an

d S

mall

(N=

52

)

Med

ium

an

dLa

rge

(N=

29

)

Very

Sm

all

an

d S

mall

(N=

17

)

Med

ium

an

dLa

rge

(N=

6)

Very

Sm

all

an

d S

mall

(N=

44

)

Med

ium

an

dLa

rge

(N=

8)

Med

ium

an

dLa

rge

(N=

24

)

Lighting Cooling Gas Appliances Other Equipment ProcessEquipment

Perc

en

t of

Eq

uip

men

t A

dop

tion

s

Don't know

No

Yes


Percent of Equipment Adoptions Recommended in the Audit Report

• Recommendations in the audit reports clearly leads to equipment adoptions, especially for lighting measures


N=52

N=32

N=69

N=190

N=49

N=37N=132

N=129

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Very Small Small Medium Large

Perc

ent

Imple

menti

ng C

onse

rvati

on M

easu

res

Participant

Nonparticipant

Implementation of Conservation Measures by Customer Size

• Counter to PY2002 evaluation results, Audit influence on conservation measure adoptions is substantial


N=139

N=51N=38

N=35

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Very Small Small Medium Large

Pe

rce

nt

of

Co

ns

erv

ati

on

Me

as

ure

s B

eg

un

as

a R

es

ult

of

the

Au

dit


Percent of Conservation Measures Begun as a Result of the Audit

• Self-reported conservation practices begun as a result of the audit ranges from 46 percent among the very small to about 25 percent for the largest customers


Customer Size

On-Site Audit ApproachVery Small(<20 kW)

Small(20 - <100 kW)

Medium(100 - <500 kW)

Large(>500 kW)

Set goals based on number of completed audits • •Implement audit outreach to Express Efficiency participants • •Set goals based on energy saving targets • •

Tailor On-Site Audit Delivery by Customer Size

• Program outreach – program goals should drive the focus of the outreach effort to obtain the desired mix of small, medium and large customers


Customer Size


Small(20 - <100 kW)

Medium(100 - <500 kW)

Large(>500 kW)

Deliver the audit report during the site visit • •Provide an Express Efficiency application; filled-in where possible • •Furnish a list of qualified service providers • • •

Tailor On-Site Audit Delivery by Customer Size (Cont’d)

• Site Visit – for smaller customers all audit services should seek to maximize energy efficient equipment adoptions and Express Efficiency participation


Customer Size


Small(20 - <100 kW)

Medium(100 - <500 kW)

Large(>500 kW)

Smaller customers need educational information on relevant measures • •Smaller customers need measures ranked by the cost to the bottom line • •Larger customers need customized information relevant to their building • •Larger customers need measures ranked by ROI to justify investment • •


• Audit Report and Recommendations – tailor audit products to customer wants and needs by customer size


Customer Size


Small(20 - <100 kW)

Medium(100 - <500 kW)

Large(>500 kW)

Track account rep. performance using completed Express applications • •In-depth follow-up efforts with larger customers to maximize savings • •


• Audit Follow-up – motivate account representatives to follow-up using job performance incentives for achieving program savings and equitable spending targets


Program Implementation for Smaller Customers

• Overall, a direct install program may be a preferable delivery channel for bringing energy efficiency to the under 20 kW market

• Alternatively, providing a turnkey program that combines Express Efficiency and Audit services may improve program results for very small customers

• Outreach efforts to small customers should emphasize remote Audits

• Emphasize low-cost and no-cost recommendations in Audits directed at smaller customers


Program Implementation for Larger Customers

• Continue to direct on-site audits to larger customers • Provide in-depth follow-up services to maximize

recommended measure uptake and support for the Express Efficiency and SPC rebate programs

• Provide well-documented, detailed return-on-investment (ROI) calculations in the Audit report

• Include significant analytic customization and site-specific research in Audit reports for large customers


Other Program Implementation Recommendations

• Provide Audit program outreach services using previous years’ Express Efficiency participants

• With the exception of SoCalGas, all other IOUs should increase Audit emphasis on the gas appliance end-use category

• Use outreach or follow-up strategies to improve customer utilization of the CD-ROM Audit tool

• Assess program marketing effectiveness and emphasize best practices

• Consider tracking goals based on downstream energy efficiency uptake

electric / gas / water summary of final evaluation report prepared by: rafael friedmann, pg&e...

Documents