do we really need large spectral libraries for the assessment of soil organic carbon at local scale?

Do we really need large spectral libraries for the assessment of soil organic carbon at local scale?

1 University Miguel Hernandez of Elche (Spain); 2 SLU (Sweden); 3 University Rey Juan Carlos (Spain); 4 CSIRO – Land & Water (Australia); 5 Cranfield University (UK); 6 University Polytechnic Cartagena (Spain);

7 University of Málaga (Spain).

César Guerrero1, Johanna Wetterlind2, Bo Stenberg2, Fernando T. Maestre3, Raphael A. Viscarra Rossel4, Abdul M. Mouazen5, Boyan Kuang5, Raúl Zornoza6, J.D. Ruiz-Sinoga7

Introduction

* Many times, we need to analyze a large number of samples collected at local scale (i.e., collected in a target site). * Options and approaches for local-scale assessment with NIR spectroscopy:

i) LOCAL models (spectrum-specific model) • Models constructed with similar spectra • Expensive: a large library is needed for select enough similar spectra

ii) Geographically local model (site-specific model) • Models constructed with similar samples (similar matrix) • Expensive approach: these models are only valid for the target site

iii) Large-scale models (regional, national, continental…) • A large model is not a guarantee of accurate predictions (large variability) • Prone to include non-linear patterns: biased predictions • We can apply “spiking”

Introduction

SOC NIR spectra

Kennard-Stone algorithm

Spiked Calibration

Spectral Library

Target Site

Prediction Set

Calibration R2, RMSEP, bias,

SEP, RPD

Spectral Library

Target Site

Prediction Set

R2, RMSEP, bias,

SEP, RPD ŷ

Spiking Subset

PLS-R PLS-R

Target site (unknown)

SOC ? SOC ?

Spiking

- Useful approach when large-scale models (regional or national) are used at local scale (in a target site). - This approach ensures that model contains similar samples (to those to be predicted). - Implies analytical efforts: the spiking subset must be analyzed with the reference method. - The spiking subset should be small: small contribution on the model (especially if the calibration is big).

PLS-R PLS-R

Unspiked Spiking

Spiking subset

Kennard-Stone algorithm

Spiked Calibration with extra-weighting

Spectral Library Target

Prediction Set

Copies

Spiking Subset

Spiking subset extra-weighted

Spiking Subset

Spiking

SOC NIR spectra

Copy Paste Paste Paste Paste

copies

Library samples

Spiking subset

cross-validation

0 2 4 6 8 10

Spiking

cross-validation

0 2 4 6 8 10

Spectral library Target

Prediction Set

Copies

Kennard -Stone algorithm

0 2 4 6 8 10

SOC NIR spectra

Copy Paste Paste Paste Paste

copies

Library samples

Spiking subset

cross-validation

the calibration is forced to fit better the spiking subset

Spiking

cross-validation

0 2 4 6 8 10

Spiking + extra-weighting

Prediction Set

Copies

Kennard -Stone algorithm

SOC NIR spectra

Target site (unknown)

PLS-R X

Spiking subset (n=8)

SOC NIR spectra

Y X Target site

(unknown)

PLS-R X

Spiking subset (n=8)

Copies = Extra weight

Unspiked calibration

Spiked calibration

Spiked calibration with Extra-weight

Unspiked Calibration

Spectral library

Target Site

Prediction Set

Spiked Calibration

Spectral library

Target Site

Prediction Set

Copies

Calibration types

Unspiked Spiking Spiking & Extra-weighting

Unspiked Calibration (UC)

Spectral library

Target Site

Prediction Set

Spiked Calibration (SC)

Prediction Set

Spiked Calibration with extra-weighting (SC+EW)

Prediction Set

Copies

Material & methods

Unspiked

National set Spain n=1096

Spectral library

Target Site

Prediction Set

Spiking Spiking & Extra-weighting

Prediction Set

Copies

Material & methods

Unspiked

Sweden n=125

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Valencia n=121

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Forest, Alicante n=130

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

UK n=104

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Forest, Murcia n=156

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Gypsipherous n=95

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Almería (mica schists) n=60

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Cuenca n=55

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Palencia n=55

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Total n=901

Spectral library

Target Site

Prediction Set

Copies

Material & methods

R2, RMSEP, bias,

SEP, RPIQ, RPD

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Sweden n=125

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Valencia n=121

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

UK n=104

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Gypsipherous n=95

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Cuenca n=55

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Palencia n=55

Total n=901

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

R2, RMSEP, bias,

SEP, RPIQ, RPD

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Sweden n=125

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Valencia n=121

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

UK n=104

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Gypsipherous n=95

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Cuenca n=55

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Palencia n=55

Total n=901

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

R2, RMSEP, bias,

SEP, RPIQ, RPD

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Predictions: national set (n=1096)

0.85 0.87

Unspiked Spiked Spiked EW

1.34 1.56

R2 RMSEP (%SOC) |Bias| (%SOC) RPIQ

Measured SOC (%)

0 5 10 15 20

Unspiked Spiked Spiked & extra-weight

0.85 0.87

1.34 1.56

R2 RMSEP (%SOC) RPIQ

Measured SOC (%)

0 5 10 15 20

Spiked Spiked & extra-weight

Measured SOC (%)

0 5 10 15 20

Unspiked

n=901 n=901

|Bias| (%SOC)

0.85 0.87

1.34 1.56

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901 n=901

|Bias| (%SOC)

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

33 % National set Spain n=362

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Measured SOC (%)

0 5 10 15 20

Predictions: 33% national set (n=362)

|Bias| (%SOC)

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901

|Bias| (%SOC)

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901 n=901

|Bias| (%SOC)

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Provincial set (Alicante) n=147

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

0.11 0.04

Predictions: provincial set (n=147)

0.96 0.97

0.66 0.55

Measured SOC (%)

0 5 10 15 20

|Bias| (%SOC)

0.11 0.04

0.96 0.97

0.66 0.55

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901

|Bias| (%SOC)

0.96 0.97

0.66 0.55

0.11 0.04

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901 n=901

|Bias| (%SOC)

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Geograph. local set (Alicante) n=88

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

0.06 0.01

Predictions: Geographical local set (n=88)

0.96 0.96

0.65 0.60

5.34 5.75

Measured SOC (%)

0 5 10 15 20

|Bias| (%SOC)

Measured SOC (%)

0 5 10 15 20

0.96 0.96

0.65 0.60

5.34 5.75

0.06 0.01

Measured SOC (%)

0 5 10 15 20

n=901 n=901

|Bias| (%SOC)

0.96 0.96

0.65 0.60

5.34 5.75

0.06 0.01

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901 n=901

|Bias| (%SOC)

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

50% Provincial set n=73

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

0.02 0.03

Predictions: 50% provincial set (n=73)

0.96 0.97

0.60 0.55

5.81 6.25

Measured SOC (%)

0 5 10 15 20

|Bias| (%SOC)

0.02 0.03

0.96 0.97

0.60 0.55

5.81 6.25

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901

|Bias| (%SOC)

0.96 0.97

0.60 0.55

5.81 6.25

0.02 0.03

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901 n=901

|Bias| (%SOC)

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

Geograph. local set #2 (Guadalajara)

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

0.04 0.01

Measured SOC (%)

0 5 10 15 20

Predictions: Geographical local set #2 (n=40)

0.95 0.96

0.70 0.66

4.94 5.28

|Bias| (%SOC)

0.04 0.01

0.95 0.96

0.70 0.66

4.94 5.28

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901

|Bias| (%SOC)

0.95 0.96

0.70 0.66

4.94 5.28

0.04 0.01

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901 n=901

|Bias| (%SOC)

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

25% Provincial set n=36

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

0.09 0.01

0.96 0.97

0.61 0.55

Measured SOC (%)

0 5 10 15 20

|Bias| (%SOC)

0.09 0.01

0.96 0.97

0.61 0.55

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901

|Bias| (%SOC)

0.96 0.97

0.61 0.55

0.09 0.01

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

Measured SOC (%)

0 5 10 15 20

n=901 n=901 n=901

|Bias| (%SOC)

Unspiked

Spectral library

Target Site

Prediction Set

Copies

Material & methods

0.96 0.95 0.96 0.96 0.96 0.90 0.87

0.97 0.96 0.97 0.96 0.97 0.96 0.96

0.03 0.00

0.17 0.18

0.85 0.85

Determination coefficient (R2)

+ Spectral library size - + Spectral library size - + Spectral library size -

Spectral library

Target Site

Prediction Set

Copies

SEP (%SOC)

2.82 4.07 4.06

2.04 2.00

0.60 0.70 0.60 0.65 0.65 1.60 1.85

0.55 0.66 0.55 0.60 0.55 0.68 0.68

Spectral library

Target Site

Prediction Set

Copies

|Bias| (%SOC)

1.56 1.65

0.09 0.04 0.02 0.06 0.11

0.95 1.23

0.01 0.01 0.03 0.01 0.04 0.20 0.21

Spectral library

Target Site

Prediction Set

Copies

RMSEP (%SOC)

3.82 4.45 4.97

2.57 2.59

0.61 0.70 0.60 0.65 0.66 1.85 2.22

0.55 0.66 0.55 0.60 0.55 0.71 0.71

Spectral library

Target Site

Prediction Set

Copies

0.39 0.16

0.83 0.71 0.63

1.23 1.22

5.30 4.87 4.81

1.70 1.42

5.70 5.24

4.47 4.46

Spectral library

Target Site

Prediction Set

Copies

0.43 0.17

0.91 0.78 0.70

1.35 1.34

5.81 5.34 5.28

1.87 1.56

6.25 5.75

4.90 4.89

Spectral library

Target Site

Prediction Set

Copies

0.43 0.17

0.91 0.78 0.70

1.35 1.34

5.81 5.34 5.28

1.87 1.56

6.25 5.75

4.90 4.89

Spectral library

Target Site

Prediction Set

Copies

Conclusions and recommendations

• When a large-sized calibration is spiked, the spiking subset should be extra-weighted. Otherwise, the impact of the spiking subset will be low (especially if the spiking subset is small). • For local scale assessment, we don’t need the development of large spectral libraries: once spiked, the small-sized performed even better than the large-sized calibrations. • Do not focus your analytical efforts exclusively in the development of large-sized libraries and large-sized models. Split your analytical efforts in:

• the development of small spectral libraries (a few regional samples) • the analysis of the spiking subsets

• This approach (spiking + extra-weight) gives accurate results and small analytical efforts are needed (new users attracted, more soils would be studied, more benefits for soil management)

Thank you very much for your attention!!

Criticisms welcomed!!

Acknowledgements This work is part of the research project (Ref. CGL2011-27001) “Estimación de carbono orgánico en suelos mediante espectroscopía de infrarrojo cercano (NIR): estrategias para el desarrollo de modelos con aplicabilidad a escala nacional” financed by the Spanish “Ministerio de Economía y Competitividad” (Plan Nacional I+D+i)

R2 |bias|

RMSEP SEP

RPD RPIQ

do we really need large spectral libraries for the assessment of soil organic carbon at local scale?

Education

monographic purchasing trends in research libraries: did...

radiochemistry webinars - wipp. · pdf filegamma...

do school libraries really make a difference? read these...

quickmod tutorial · liberator: creating spectral libraries...

lipidomics quantitative (mrm) methods, mass …...lipidomics...

open library environment designing technology for the way...

analysis of large scale soil spectral libraries

analysis of large scale soil spectral libraries€¦ ·...

learning infused libraries: honest talk about what it...

newsletter - answer-itn · structural elucidation and...

what libraries can learn from kanye · libraries can learn...

skyline small molecule multidimensional spectral libraries

synthetic spectral libraries for gaia vallenari inaf,...

social media collaboration @ law libraries · social media...

cq4. ecosystem function and diversity · 2012. 1. 5. ·...

sait teramo – may 6 th, 2008 antonella vallenari spectral...

surface cover spectra of the big pine creek watershed ·...

an ala washington office webinar make a difference for...

as toolbox & processing of field spectra · 1.go to envi...

undergraduate research project university of rochester river...