Scientific Irrigation SchedulingSubcommittee Update

Regional Technical ForumJune 2, 2014

2

Overview

• RTF Scientific Irrigation Scheduling (SIS) measure has been out of compliance with the Guidelines since the Guidelines were developed three years ago.

• Attempts by RTF contractors (2012) and RTF Staff (2013) to update the measure to a Standard Protocol have been unsuccessful.– Challenges in determining both the baseline and efficient cases– Available baseline data insufficient

• RTF Staff have discussed possible approaches to updating the measure with regional experts and stakeholders and still don’t have a feasible path forward.

• BPA is researching the feasibility of a method to estimate savings, which is based on the statistical methods that they are currently using for custom industrial measures.

• For now, Staff and BPA would like to provide the Subcommittee and the RTF with: – Update of efforts and findings to date– Options going forward, and Subcommittee recommendations

3

Today’s Objectives

1) Review recent RTF history of the measure2) Discuss the (in)feasibility Staff-suggested research

plans3) Identify additional research plan options4) Discuss the options that the RTF has for handling

this measure and collect Subcommittee recommendations on these options.

4

1) Review recent RTF history of the measure2) Discuss the (in)feasibility Staff-suggested research

plans3) Identify additional research plan options4) Discuss the options that the RTF has for handling

this measure and collect Subcommittee recommendations on these options.

5

Recent History of Measure• Subcommittee meetings – July 9, 2013 / August 8, 2013 / November 8, 2013

– Reviewed current measure, discussed Guidelines compliance issues, discussed feasibility of research

– Meeting minutes and supporting documents: http://rtf.nwcouncil.org/subcommittees/SIS/meetings.htm

• RTF Presentation – Provisional Standard Protocol proposal – November 19, 2013– RTF Decision: refer the entire measure to the SIS Subcommittee to evaluate the

baseline conditions and the potential inclusion of a screen based on crop type and contract type.

• Subcommittee Survey – retrofit or current practice baseline? – December 2013– Conclusions: SIS is a current practice measure because every year a grower “shops”

for an irrigation strategy, and no SIS equipment is left in place for the next year.• Staff and BPA visit to IRZ Consulting - January 23, 2014

– Discussed technical details of SIS, the RTF’s challenge, and problems with proposed research options

6

Recent History of Measure (cont’d)• Staff conclusions:– Existing data is not sufficient for a proven Standard

Protocol– A feasible research plan to obtain reliable baseline

data for a provisional Standard Protocol has not been identified

• BPA agrees to look into bringing measure into compliance with the Guidelines – January, 2014

7

1) Review recent RTF history of the measure2) Discuss the (in)feasibility Staff-suggested research

plans3) Identify additional research plan options4) Discuss the options that the RTF has for handling

this measure and collect Subcommittee recommendations on these options.

8

Proposed Research Approaches• RTF Staff have discussed several possible research

approaches to obtain reliable baseline data with the Subcommittee and other regional experts

• The following slides detail several proposed methods that staff believes would result in a proven RTF measure

9

Proposed Research Approaches• Pre/post study of participants, estimating water

consumption from electric meter records– Pros:

• Baseline Data already exists and/or could be collected via survey

– Cons: • Would only work on “simple systems” – pump/pumps on a

dedicated meter and only serving a single crop. These tend to be small farms, and wouldn’t be representative of the market.

• Would require sample size of 100s to 1000s simple systems which is unlikely to be obtained

– BPA is looking into the use of regression methods for “complex” systems.

10

Proposed Research Approaches• Phone survey (similar to Quantec, Phase 1) – ask

about irrigation practices by crop type, and water consumption by crop type.– Pros:

• Could be used to determine the baseline portions of the market using efficient and inefficient practices

• Inexpensive– Cons:

• Couldn’t be used for savings percentage: Subject matter experts do not consider self-reported water consumption number reliable for non-participants.

11

Proposed Research Approaches• Custom query of USDA Farm and Ranch Irrigation

Survey (FRIS) data – look at irrigation decision practices and water applied per crop type.– Pros:

• Could be used for an inexpensive, crude estimate of baseline irrigation practices

– Cons: • FRIS asks for irrigation practices at the farm level, not more

granularly at the crop-type level. – Could assume that irrigation practice applies to most prevalent

(acreage) crop type at each farm.• Subject matter experts do not consider self-reported water

consumption number reliable for non-participants.• Would need to know soil type to estimate Water Requirement

12

Proposed Research Approaches• Metered study (water consumption, at the

field level) of growers with and without SIS.– Pros: • Would provide the most accurate estimate of savings

– Cons:• Sample size needed get a reliable estimate would be

100s to 1000s of sites, which is unlikely to be obtained. Sample size increases for results by crop type, farm size, etc.• Cost of study would be prohibitive.• Recruiting study participants would be challenging

13

1) Review recent RTF history of the measure2) Discuss the (in)feasibility Staff-suggested research

plans3) Identify additional research plan options4) Discuss the options that the RTF has for handling

this measure and collect Subcommittee recommendations on these options.

14

Additional Research Plan Options• [BPA Presentation]

15

Additional Research Plan Options• [Subcommittee discussion]

16

1) Review recent RTF history of the measure2) Discuss the (in)feasibility Staff-suggested research

plans3) Identify additional research plan options4) Discuss the options that the RTF has for handling

this measure and collect Subcommittee recommendations on these options.

17

Options for the RTF - Proven• Proven: RTF accept the Quantec study results as

Best Practice and sufficiently reliable– Use Quantec Phase 2 results – SIS saves 10% relative

to inefficient irrigation practices– Use Quantec Phase 1 results - 57% of acreage in the

baseline is subject to inefficient irrigation practices• 57% of acres irrigated without water management (10%

savings)• 43% of acres irrigated with water management (0%

savings)– [Water Savings %] = [10%]x[57%] + [0%]x[43%] = 5.7%

18

Options for the RTF – Proven (cont’d)Guidelines issues regarding accepting the Quantec study for a proven measure:• Savings Reliability - “…the sampling, data collection and other errors are

sufficiently small and unbiased. Sufficiency is determined by the collective opinion of the RTF.” (Guidelines, Roadmap Section 1.3.6)

• Best Practice Savings Estimate – Estimate relies on the “best practical and reliable data collection and estimation methods.” Practical = can be carried out with proven techniques and resources deemed reasonable by the RTF.” (Guidelines, Roadmap Section 1.3.8. )

• RTF would need to agree that:– The uncertainty in the Quantec 10% savings estimate is “sufficiently small and unbiased”.

• This would be a tough sell – The Quantec study remains the best practical and reliable data collection and estimation

method.• However, if this is true, then repeating the study to obtain a larger sample seems like a

feasible option.– Measure identifiers such as crop type and location are not necessary

Or– Crops that are commonly subject to efficient practices and areas that are subject to water

constraints are ineligible for the measure

19Options for the RTF - Provisional• Provisional: • Option 1: Regression-based, metered study – both simple and complex systems

– Research plan development led by BPA with support of RTF Staff / SIS Subcommittee / R&E Subcommittee. Regression capable of addressing multiple crop types and pumping systems per site.

• Option 2: Regression-based, metered study – simple systems only– Limit research and measure eligibility to “simple” systems – pumping system on a single

and dedicated meter, serving a single crop type. • Option 3: Focus research on determining current practice

– Accept Quantec Phase 2 findings (10%) as sufficiently reliable– Focus research on current mix of efficient and inefficient practices by crop type, location,

and other measure identifiers• Could be done via phone survey• Could be done crudely/cheaply via USDA FRIS

• Any provisional option will require a provisional savings estimate– Staff recommends using the Quantec-based 5.7% savings derived earlier in this

presentation.– RTF could decide that the current 10% savings is a more appropriate provisional number

because it results in program willingness-to-pay high enough to attracts participants.• Staff does not recommend this since it is known that some level of efficient irrigation practices

exist in the region, and choosing 10% ignores that fact.

20

Options for the RTF - Deactivate• Deactivate: – Acknowledge that while SIS is a useful service and most

likely results in energy savings, it would be too expensive to determine these savings with the level of certainty required by the RTF for a Proven measure.

– Utilities could continue to offer SIS in the region, but without RTF approval.

21

[Discussion]

22

Back-up Slides

23

Extra Material:A review of the current measure and research basis

24

Overview of Measure• “Scientific irrigation scheduling is a process growers of agricultural

products can use to improve irrigation water management. When used properly, scientific irrigation scheduling provides information on when to irrigate, how much water to apply, and how to apply water to satisfy crop water requirements and avoid plant moisture stress. When used appropriately, irrigation scheduling saves water, energy, labor, and fertilizer, and in many cases improves crop yields and crop quality.”– http://www.bpa.gov/energy/n/agriculture.cfm (emphasis added)

• Uses soil moisture monitors, and modeling specific to crop type, soil type, and local meteorology to determine when/how much to water.

• Traditional methods rely more on look/feel of soil and crops, predetermined watering calendars, and water availability.

25Overview of Measure: Energy Savings• 1) Ex post estimate water savings (acre-ft) at end of

season – SIS service provider reports how much water was used for irrigation– [Water savings] = [Water consumption] x [10%]

• 2) Estimate energy intensity (kWh/acre-ft), as a function of pump system at site: – Total Dynamic Head (a function of Lift and Discharge Pressure)– Pump System Efficiency estimates (a function of Pump Type, Pumping

Plant Size, and Lift)

• 3) Estimate energy savings (kWh)– [Energy Savings] = [Water Savings] x [Energy Intensity] = [Water Consumption] x [10%] x [Energy Intensity]

Where does the 10% come from?

26

Quantec StudyThe “10%” current savings estimates are based on research conducted by Quantec from 2003 - 2005• Quantec, “A Study of Irrigation Scheduling Practices

in the Northwest” December 2003, for BPA.• Quantec, “Phase II: Measurement of Water and

Electricity Impacts”, June 2005, for BPA, NEEA, Pac. NW Generating Cooperative

27

Quantec Study – Phase 1. Baseline Practices

43% of irrigated acres are irrigated efficiently– Survey of 776 farms across the PNW– Quantec categorized farms according to irrigation decision-making

practices• “Examination of reported water

use and their deviations from known irrigation requirements indicated that, by and large, farms in practice level I tend to use less water than farms that use less sophisticated practices. Comparison of mean water use derived from a regression model of water use showed that application of the combination of methods used in practice levels I and II are likely to result in water savings of approximately 12% and 10%, respectively.”

28

Quantec Study – Phase 2. Impact Study

The impact study compared growers known to use outside SIS services to growers known not to practice water management. I.e. – compare the best to the worst.

“It was therefore decided to select the treatment group from among growers who received water management services through GWMA or IRZ Consulting, and to select the control group from farms in close proximity to the treatment farms. The main advantage of this approach was that it offered a more consistent basis for defining water management practices among the treatment group and significantly helped the recruitment and data collection processes.

“To ensure comparability with the treatment group, each treatment field was matched with a local control field with the same crop grown by a farmer known not to practice water management.”

29

Quantec Study – Impact Study• Analysis

– Meter irrigation water use - indirect measurement: log line pressure at point of delivery, multiply by sprinkler design flow rate

– Estimate “ideal” water use – use water balance model specific to county-level weather, field soil type, crop type (evapotranspiration rates)

– Ideal water use is the normalizing factor for all fields (control and treatment)

• Findings– Control group used 12% more water than ideal– Treatment group used 2% more water than ideal– Based on this, a 10% water savings for SIS is proposed

30

Quantec Study – Impact Data

Note:• Small sample size• Limited crop type• High variance

31

Quantec Study – Impact Data

Variance = ( [actual] – [ideal] ) / [ideal]

Impact appears to be on the least efficient irrigators.

32

Quantec Study – Limitations of Analysis• Small sample size – 19 treatment, 19 control

– Results not statistically significant– But SIS subcommittee and regional experts have said that 10% is a

reasonable estimate of savings for growers who weren’t using efficient methods before.

– Others have suggested that savings are as high as 20%.• If we only know that our Best Practice savings are between 0 and 20%, we can’t say if

our 10% Simplest Reliable estimate is within 10% of the Best Practice estimate.

• Limited geography and crop type• Dated – study conducted 2003 – 2005• Baseline only represents a subset of the population (non-SIS

users)• Does not examine take-back – (i.e., use of saved water

elsewhere on farm)

33

Extra Material:Why is the current measure out of compliance with the RTF Guidelines?

34

Guidelines Compliance IssuesA Proven Standard Protocol requires• Best Practice method of estimating savings

– Best practice method would require direct (water) or indirect (kWh) metering of treatment and control sites

– We have some of this data from the Quantec study, but it is neither current or statistically significant.

• Simplest Reliable method that is sufficiently reliable (Guidelines, Savings, Section 3.4.2)– With +/- 20% of Best Practice per site, or +/-10% overall– The 10% of Water Requirement is a candidate Simplest Reliable method.– Because we don’t have a Best Practice savings estimate, we can’t say if

our 10% estimate meets this criteria.

35

Guidelines Compliance IssuesA Proven Standard Protocol requires (cont’d)• Current Practice or Pre-conditions baseline– Current Practice:

• SIS subcommittee indicated that SIS is best described as a current practice measure: each year growers are in the market for an irrigation strategy. There is no SIS infrastructure left in place at the site.

• Savings for a current practice SIS measure could be claimed repeatedly.

• A current practice baseline “is defined by the recent typical choices of eligible end users in purchasing new equipment and services” (Roadmap, Section 3.2.1)– This would include both inefficient and efficient irrigation practices.– The 10% savings estimate only includes inefficient irrigation practices in the

baseline

36

Guidelines Compliance IssuesA Proven Standard Protocol requires (cont’d)• Current Practice or Pre-conditions baseline– Pre-Conditions:

• If SIS were seen as a behavioral change, with impacts lasting more than one growing season, then a pre-conditions baseline would be appropriate.

• For the pre-conditions baseline, the 10% savings estimate (if it were sufficiently reliable), would be appropriate.

• However, savings could not be claimed repeatedly, because once SIS was implemented at a site, the pre-condition at that site in future years would be SIS.

• Effectively, the program would be limited to sites that had not previously used SIS or similar, efficient irrigation practices.

37

Guidelines Compliance IssuesA Proven Standard Protocol requires (cont’d)• Measure identifiers (Roadmap, Section 3.1)

– Measure identifiers should be used to uniquely identify measure applications where there are large differences in savings

– The current measure applies the same savings 10% in all cases.– However, we have heard from the Subcommittee that savings

depend on• Crop type – Current practice varies by crop type• Water availability and rights – savings primarily occur where water consumption

is not constrained. The frequency of water constraint varies by area (e.g. some areas have more water scarce years than others).

• Prevalence of SIS in territory – Infrastructure and local adoption drive additional adoption.

38

Recent History of Measure• http://rtf.nwcouncil.org/measures/measure.asp?id=184• 2006 - Deemed Calculator approved, still in use• 2012 - Cascade contracted to update measure to comply with guidelines for standard protocol• Nov. 2012 – Cascade presentation to RTF

– Meeting minutes: http://rtf.nwcouncil.org/meetings/2012/11/minutes20121114.pdf– Excerpts:

“Jones made a motion that the RTF approve the Scientific Irrigation Scheduling Standard Protocol and move it to the “Proven” category with a sunset date of November 2017. Harris seconded the measure. Hadley said he would vote against the motion because it does not comply with the guidelines. We are using the 10 percent savings value within the best practice method, and we can’t test that number, he stated.

“Lauren Gage also expressed concerns about the calculator and said she would vote against the motion. Hope said he agreed with Hadley and would vote against the motion. The RTF discussed the data and concerns about whether it conforms to the guidelines.

“Hadley made a motion to table a decision on the SIS protocol until the RTF adopts new guidelines that address data source requirements for parameters used within standard protocols. Jones seconded the motion, which passed with 20 votes in favor and none against.”

• April 2013 – RTF approved changes to Guidelines. The Guidelines state that the RTF ultimately decides on the validity of a data source for a parameter after “Diligent Review.”