model comparison for short-term growth projections on a western oregon property

Natural Resource Consultants Since 1921

Model ComparisonModel Comparisonfor short-term growth for short-term growth

projections on a Western projections on a Western Oregon propertyOregon property

Model ComparisonModel Comparisonfor short-term growth for short-term growth

projections on a Western projections on a Western Oregon propertyOregon property

GMUG MeetingGMUG Meeting

June 2, 2010June 2, 2010


G&Y Models G&Y Models SurveySurvey

* PACIFIC NORTHWEST TIMBERLAND INVESTMENT SURVEY RESULTS, MARCH 2010, SIZEMORE & SIZEMORE, INC.

• 15 Respondents (Industry + TIMOs + Consultants), multiple responses allowed

G&Y Model Used Percent

SPS/FPS 8 28%

FVS 7 24%

ORGANON 5 17%

In-House 5 17%

Other 2 7%

Excel 1 3%

DFSim 1 3%

*


Data - LocationData - Location

• Western Oregon

• South of Portland, North of Eugene

• Stands in Valley & up west slope of Cascades


Data - RemeasurementData - Remeasurement

Interval (Yrs) No Remeasurement No Thin Thinned* Grand Total

0 16 2 18

3 2 2

4 1 1

5 17 13 30

6 2 2

7 2 2

8 3 3

9 1 1 2

10 52 52

13 1 1

Grand Total 16 81 16 113

Count of Plots:


Data – Age & SIData – Age & SICount of Plots:

Site Category

Age Class HIGH MED LOW Grand Total

15 - 24 7 9 0 16

25 - 34 6 4 1 11

35 - 44 16 6 0 22

45 - 54 18 13 3 34

55 - 64 2 5 0 7

65 - 74 0 2 0 2

Grand Total 49 39 4 92


Site ClassesSite Classes

Site Category Site Index Assigned SI*

High Greater than 120 125

Medium Greater than 95 and less than or equal to 120

110

Low Less than or equal to 95 95

* Prior to 1997, SI class assigned based on growth rings


Stand Statistics (Init)Stand Statistics (Init)

TPA BA QMD Avg Ht Ht40 BFV

Count 92 92 92 92 92 92

Mean 238 164 11.7 74 87 20,490

Min 75 17 5.2 28 29 0

Max 620 289 20.1 121 129 48,129

StdDev 101 61 3.5 24 27 13,682

StdErr 10.6 6.4 0.36 2.5 2.8 1,426

StdErrPct 4.4% 3.9% 3.1% 3.4% 3.3% 7.0%


Merch SpecsMerch SpecsSpecification Value

Minimum DBH 7 inches

Stump Height 1 foot

Log Length 32 feet

Minimum Top DIB 5 inches

Minimum Log Length 16 feet

Log Rule Long Log

Trees Must Contain 1 Full Log No

Trim Amount 12 inches - fixed

Use Scribner Decimal C No

Hidden Defect 0


Tree StatisticsTree Statistics

• 4,382 trees across 15 species– DF (3,469), WH (604)


Growth ModelsGrowth Models

• SIS (SPS)

• FPS (v6.73)

• FVS (12/16/2008)– PN, WC

• ORGANON (v8.2)– NWO, SMC


Growth Models - OptionsGrowth Models - Options

• SIS (SPS)– Client coefficient file

• FPS – Library 11

• FVS– Max SDI: DF 600, WH 720, RA

300

• ORGANON (v8.2)– Max DF SDI = 600


Comparison MethodsComparison Methods

• Measured vs. Predicted

Average Growth

• Basic Error Statistics

• Error CDF’s

• Error Significance Tests

• Linear Regressions of Errors

• 50-Year Growth Projections


Comparison CaveatComparison Caveat

• Period length is not uniform across plots

• The same set of data was used across models

• Relative size of errors important


Measured vs. Predicted Measured vs. Predicted Average GrowthAverage Growth

• High level view of growth

• PAI

• Focus on four youngest age classes due to sample size



• Performance?– SIS, PN, FPS, NWO all about

same so far– SMC not as good– WC least accurate so far


Basic Error Stats - StandsBasic Error Stats - Stands

• Errors: Predicted - Observed– Negative = under-prediction– Positive = over-prediction

• Errors not annualized**

• Stand-level and Tree-level

• FPS – no grown treelist output


Basic Error Stats - StandsBasic Error Stats - Stands SIS FPS FVSPN FVSWC O-SMC O-NWO

TPA Mean 2.3 8.8 -3.5 6.8 10.6 7.1

StdDev 22.3 19.5 20.7 21.5 19.4 20.3

StdError 2.325 2.035 2.159 2.242 2.026 2.112

BA (ft2/ac)

Mean -1.6 -1.1 3.6 -21.3 -8.5 -4.5

StdDev 17.4 15.3 26.5 14.3 15.3 13.8

StdError 1.809 1.598 2.758 1.491 1.596 1.434

QMD (inches)

Mean -0.1 -0.4 0.3 -0.9 -0.6 -0.4

StdDev 0.71 0.52 0.91 0.57 0.46 0.47

StdError 0.074 0.054 0.095 0.059 0.048 0.049

HT40 (feet)

Mean -1.7 na -2.1 -8.5 -2.8 -2.6

StdDev 4.2 na 3.9 4.8 3.7 3.9

StdError 0.438 na 0.410 0.500 0.381 0.402

BFVOL (Bdft/ac)

Mean -1,426 -3,289 -1,581 -6,489 -2,596 -1,811

StdDev 3,397 3,425 4,233 4,008 2,925 2,557

StdError 354.1 357.1 441.3 417.9 305.0 266.6


Basic Error Stats - StandsBasic Error Stats - StandsTPA

SIS

FPS

FVSPN

FVSWC

O-SMC

O-NWO

-12

-8

-4

0

4

8

12

BA (sqft/ac)

SIS FPS

FVSPN

FVSWC

O-SMCO-NWO

-30

-20

-10

0

10

20

30

QMD (in.)

SISFPS

FVSPN

FVSWC

O-SMC

O-NWO

-1.0

-0.5

0.0

0.5

1.0

BFVol (bdft/ac)

SISFPS

FVSPN

FVSWC

O-SMCO-NWO

-8000

-6000

-4000

-2000

0

2000

4000

6000

8000

HT40 (ft.)

SIS FVSPN

FVSWC

FPS

- not

ava

ilabl

e

O-SMC O-NWO

-10

-5

0

5

10

FPS

- not

ava

ilabl

e

-10

-5

0

5

10


Basic Error Stats - TreesBasic Error Stats - Trees

SIS FPS FVSPN FVSWC O-SMC O-NWO

DBH

Mean 0.1 NA 0.7 -0.5 -0.4 -0.2

SD 0.9 NA 1.0 0.9 0.8 0.7

Std Err 0.014 NA 0.017 0.015 0.014 0.012

HEIGHT

Mean -0.6 NA -1.2 -6.5 -2.4 -2.5

SD 6.6 NA 7.5 8.0 6.6 6.5

Std Err 0.107 NA 0.123 0.130 0.107 0.106

BFVOL

Mean -8.5 NA 2.6 -32.3 -18.6 -12.1

SD 38.8 NA 36.0 48.9 37.3 32.4

Std Err 0.675 NA 0.626 0.849 0.649 0.562


Basic Error Stats - TreesBasic Error Stats - TreesDBH Errors

SIS

FVSPN

O-SMC

O-NWO

FVSWC

-1.0

-0.5

0.0

0.5

1.0

Height Errors

SIS FVSPN

FVSWC

O-SMC O-NWO

-10

-5

0

5

10

BFVol Errors

SIS

FVSPN

FVSWC

O-SMCO-NWO

-40

-20

0

20

40


Basic Error Stats - TreesBasic Error Stats - TreesDF: DBH Errors

SIS

FVSWC

O-NWO

FVSPN

O-SMC-1.0

-0.5

0.0

0.5

1.0

DF: Height Errors

SIS FVSPN

FVSWC

O-SMC O-NWO

-10

-5

0

5

10

DF: BFVol Errors

SIS

FVSPN

FVSWC

O-SMCO-NWO

-40

-20

0

20

40

WH: DBH Errors

SISFVSPN

FVSWC

O-SMC

O-NWO

-1.0

-0.5

0.0

0.5

1.0

WH: Height Errors

SISFVSWC

O-SMC

FVSPN

O-NWO

-10

-5

0

5

10

WH: BFVol Errors

SISFVSPN

FVSWCO-NWO

O-SMC

-40

-20

0

20

40


Basic Error StatsBasic Error Stats

• Performance?– SIS smallest mean errors for all

but BA (2nd smallest) and average variability

– PN probably second best choice– NWO was least variable– SMC and WC falling out of the

race


Error CDF’s - StandError CDF’s - Stand

• Cumulative Distribution Functions

• Picture of error distribution by error size

• More information than a mean bias



Cumulative Distribution of Prediction Errors: BFVol

0%

20%

40%

60%

80%

100%

<= 5% <=10% <= 15% <= 20% <= 25% <= 100%

% Error in BdVol Prediction

% o

f Plo

ts

SIS

FPS

FVSPN

FVSWC

O-SMC

O-NWO


Error CDF’s - TreeError CDF’s - Tree



Cumulative Distribution of Prediction Errors: Tree BFVOL

0%

20%

40%

60%

80%

100%

<= 5% <=10% <= 15% <= 20% <= 25% <= 100%

% Error in Tree BFVol Prediction

% o

f Tre

es

SIS

FPS

FVSPN

FVSWC

O-SMC

O-NWO



• Performance?– NWO and SIS had largest % of

small errors– FPS and SMC next best, with

PN close behind– WC doesn’t seem to be a good

fit


Significance TestsSignificance Tests

• Paired t-test vs. Wilcoxon signed rank test– All data– 5-7 yr vs. 8-10 yr Projections– Initial Vol <= 25 MBF vs. Initial

Vol > 25 MBF– Unthinned vs. Thinned



p-values SIS FPS FVSPN FVSWC O-SMC O-NWO

TPA

All 0.2459 0.9503 0.0278 0.1497 0.6656 0.2015

5-7 yr Projections 0.8441 0.0529 0.0077 0.3755 0.5902 0.3088

8-10 yr Projections 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000

Initial Vol < 25 MBF 0.0000 0.0506 0.0188 0.0050 0.0479 0.0188

Initial Vol > 25 MBF 0.0187 0.0061 0.4675 0.3694 0.0293 0.3786

UnThinned 0.5254 0.5550 0.1208 0.2195 0.8675 0.2408

Thinned 0.1981 0.2209 0.0480 0.5509 0.1981 0.5936




BA

All 0.1610 0.0398 0.0273 0.0000 0.0000 0.0023

5-7 yr Projections 0.0266 0.1347 0.0023 0.0000 0.0049 0.0686

8-10 yr Projections 0.0000 0.0000 0.0000 0.0000 0.0000 0.0557

Initial Vol < 25 MBF 0.5641 0.4096 0.3630 0.0000 0.0718 0.0067

Initial Vol > 25 MBF 0.1099 0.0673 0.0523 0.0001 0.0053 0.1687

UnThinned 0.6885 0.4298 0.1894 0.0000 0.3277 0.0105

Thinned 0.0157 0.0029 0.0008 0.0000 0.0157 0.6832




QMD

All 0.0525 0.0001 0.5513 0.0000 0.0000 0.0000

5-7 yr Projections 0.5668 0.2485 0.8441 0.0000 0.0000 0.0006

8-10 yr Projections 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000

Initial Vol < 25 MBF 0.8861 0.0107 0.1361 0.0000 0.0000 0.0073

Initial Vol > 25 MBF 0.0006 0.0001 0.0858 0.0000 0.0000 0.0000

UnThinned 0.2140 0.0013 0.3793 0.0000 0.0000 0.0000

Thinned 0.0843 0.0157 0.3967 0.0010 0.0010 0.0029




Height

All 0.0002 NA 0.0000 0.0000 0.0000 0.0000

5-7 yr Projections 0.9659 NA 0.4780 0.0000 0.2089 0.2701

8-10 yr Projections 0.0029 NA 0.0001 0.0000 0.0000 0.0000

Init Vol < 25 MBF 0.0049 NA 0.0002 0.0000 0.0000 0.0105

Init Vol > 25 MBF 0.4780 NA 0.1486 0.0000 0.0010 0.0008

UnThinned 0.0008 NA 0.0000 0.0000 0.0000 0.0000

Thinned 0.6378 NA 0.7299 0.0003 0.7299 0.8261




BD FT VOL

All 0.0987 0.0000 0.0005 0.0000 0.0000 0.0000

5-7 yr Projections 0.5214 0.0005 0.0019 0.0000 0.0000 0.0000

8-10 yr Projections 0.0154 0.0000 0.0200 0.0000 0.0000 0.0000

Initial Vol < 25 MBF 0.9414 0.0000 0.7521 0.0000 0.0000 0.0184

Initial Vol > 25 MBF 0.0000 0.0000 0.0000 0.0000 0.0544 0.0004

UnThinned 0.3767 0.0000 0.0065 0.0000 0.0000 0.0000

Thinned 0.0157 0.0010 0.0132 0.0000 0.0132 0.0076



• Performance so far?– Shorter projections generally

better (higher p-values)– Less significance in higher

volume stands than lower– Thinned vs. unthinned unclear– SIS and PN least significant

differences, followed by NWO


Simple Linear RegressionSimple Linear Regression

• Oi = β0 + β1*Pi + εi

• Nature of the bias if it is present, telling us whether it is constant or changing and to what degree.


SLR ResultsSLR ResultsTPA BA QMD HT BFVol

Model Parameter Value Sig Value Sig Value Sig Value Sig Value Sig

SISIntercept 7.45 No 21.98 Yes 1.53 Yes 0.94 No 1999.39 Yes

Slope 0.9763 No 0.8852 Yes 0.8779 Yes 0.9745 No 0.8902 Yes

FPSIntercept 12.18 Yes 5.08 No 0.46 Yes ------ --- 926.49 No

Slope 0.9844 No 0.9698 No 0.9404 Yes ------ --- 0.8649 Yes

FVS-PNIntercept 6.39 No 79.12 Yes 2.74 Yes 1.56 No 4830.75 Yes

Slope 0.9545 No 0.6326 Yes 0.8221 Yes 0.9646 Yes 0.7945 Yes

FVS-WCIntercept 6.87 No 7.32 No 0.22 No -11.43 Yes -571.86 No


O-SMCIntercept 13.86 Yes 3.87 No 0.18 No -1.99 No -348.21 No


O-NWOIntercept 11.02 Yes -0.42 No -0.01 No -0.96 No -395.82 No

Slope 0.9821 No 0.9801 No 0.9703 No 0.9837 No 0.9546 YesSignificance at α = 0.05


SLR ResultsSLR Results

• Performance so far?– NWO did best, especially when

recall low variability– FPS and SMC the next best– SIS and PN several significant

slopes and intercepts


Long-Term ProjectionsLong-Term Projections

• Long-term remeasurement data uncommon

• Grow young stands and plot against older inventory data

• Less for identifying a ‘best’ so much as pointing out models to avoid



• Performance so far?– NWO, SMC, FPS all reasonable in

long-term predictions– PN and SMC saw affect of Max

BA limit– SIS main point of concern for

long-term predictions


Overall PerformanceOverall Performance

• No Clear Winner– Basic stats and significance tests

point to SIS– CDF’s and regressions point to

NWO– Overall, SIS, PN, and NWO

comparable– FPS not far behind– SMC and WC lower performance


Questions?Questions?


Remeasurement IssuesRemeasurement Issues

IssueNumberOf Trees Types of Cruiser Comments

No Initial Height 28 unable to get height

No Final DBH 9 heavy leaner, unable to measure

No Final Height 28 top bent, leaner, broken top

Shrinking DBH 17

Shrinking Height 150 broken top

Died 373

Grand Total 605

model comparison for short-term growth projections on a western oregon property

Documents