evaluating the validity of spectral calibration models for quantitative analysis following signal...

Anal Bioanal Chem (2012) 404:2317–2327DOI 10.1007/s00216-012-6364-1

ORIGINAL PAPER

Evaluating the validity of spectral calibration modelsfor quantitative analysis following signal preprocessing

Da Chen · Edward Grant

Received: 26 March 2012 / Revised: 7 August 2012 / Accepted: 15 August 2012 / Published online: 4 September 2012© Springer-Verlag 2012

Abstract When paired with high-powered chemomet-ric analysis, spectrometric methods offer great promisefor the high-throughput analysis of complex systems.Effective classification or quantification often relieson signal preprocessing to reduce spectral interferenceand optimize the apparent performance of a calibra-tion model. However, less frequently addressed by sys-tematic research is the affect of preprocessing on thestatistical accuracy of a calibration result. The presentwork demonstrates the effectiveness of two criteria forvalidating the performance of signal preprocessing inmultivariate models in the important dimensions ofbias and precision. To assess the extent of bias, weexplore the applicability of the elliptic joint confidenceregion (EJCR) test and devise a new means to evaluateprecision by a bias-corrected root mean square error ofprediction. We show how these criteria can effectivelygauge the success of signal pretreatments in suppressingspectral interference while providing a straightforwardmeans to determine the optimal level of model com-plexity. This methodology offers a graphical diagnosticby which to visualize the consequences of pretreatmenton complex multivariate models, enabling optimizationwith greater confidence. To demonstrate the applica-tion of the EJCR criterion in this context, we eval-uate the validity of representative calibration models

D. ChenState Key Laboratory of Precision Measuring Technologyand Instruments, Tianjin University, Tianjin 300072, Chinae-mail: [email protected]

E. Grant (B)Department of Chemistry, University of British Columbia,Vancouver, BC V6T 1Z1, Canadae-mail: [email protected]

using standard pretreatment strategies on three spec-tral data sets. The results indicate that the proposedmethodology facilitates the reliable optimization of awell-validated calibration model, thus improving thecapability of spectrophotometric analysis.

Keywords Raman spectroscopy · Spectrochemicalanalysis · Feature selection · Precision · Bias

Introduction

Spectrophotometric analysis has long provided a pow-erful analytical tool with a capacity to measure a greatmany chemical and physical properties [1–6]. For com-plex systems, quantitative spectrophotometric meth-ods often depend on multivariate calibration modelsto extract accurate analytical information [7–9]. Theconstruction of a multivariate calibration model usuallystarts with spectra obtained from a set of calibrationsamples for which one has values of a quantity of inter-est determined by a reference method. The most com-monly used multivariate calibration technique is partialleast squares (PLS) [10]. PLS ignores the variancesfrom sources other than the ones of interest that haveinfluence in the same spectral regions [11, 12]. As aconsequence, spectral interference arising from instru-mental and environmental changes, as well as matrixvariation, can skew the results of a PLS analysis [13].Not only can spectral interference increase the errorof a PLS prediction, it can also introduce significantbias [14]. For this reason, one often pre-processes asignal before applying PLS to improve the calibrationperformance.

2318 D. Chen, E. Grant

A number of signal preprocessing strategies canserve as a means to reduce spectral interference [7, 8,15–17]. Most preprocessing strategies conform with oneof three general approaches [18]:

(1) Geometric spectral preprocessing, such as mul-tiplicative signal correction (MSC) [19], first-derivative filtering with Savitzky–Golay smooth-ing (SG-1D) [20], second-derivative filtering withSG-smoothing (SG-2D), discrete wavelet trans-form (DWT) [21], and continuous wavelet trans-form (CWT) [22];

(2) Dimensionality reduction methods representedby orthogonal projection and variable selectionmethods, such as orthogonal signal correction(OSC) [15] and uninformative variable elimina-tion (UVE) [23]; and

(3) Geometric and dimensionality reduction methodsapplied in combination.

To date, studies advancing these approaches have fo-cused on the methodology of signal preprocessing itself,while paying less attention to the question of validation,proving whether a calibration model based on signalpreprocessing is applicable. When undertaken, suchan examination typically applies conventional criterionfor evaluating calibration models, such as assessing theroot mean squared error of prediction (RMSEP), whichcompares the absolute fit of the measured data pointsto the model-predicted values.

This approach is not entirely satisfactory because,together with its precision, one would like to knowwhether a prediction presents significant bias, whichis to say whether preprocessing has appropriately sup-pressed the existing spectral interference and not intro-duced systematic or random distortion in other respects[14]. This is not possible if only an estimate of averageprediction error is available [24].

Further complicating this task, the conclusionsoffered by different authors regarding the applicationof certain spectral pretreatments, such as MSC, OSC,and DWT, depend on the nature of the measurementand cannot be generalized to other data sets [21, 22, 25].It is therefore very important to design an appropriategeneral criterion for validating the performance of cal-ibration models employing signal preprocessing.

Useful approaches exist to test, in particular,whether intrinsic bias affects the prediction of a multi-variate model. Here, we demonstrate the application ofone such method, which has been developed originallyfor the comparison of measurement results and thetransferable estimation of measurement uncertainty.Examining the elliptic joint confidence region (EJCR)[26–28], we systematically explore the extent to which a

selected set of pretreatment methods either suppressesor introduces bias in a calibration model.

The present work develops a criterion to validatePLS models that include signal preprocessing by mak-ing particular reference to the parameters of biasand precision. We show how to assess prediction biasby applying the EJCR test and evaluate predictionprecision by determining the bias-corrected RMSEP(RMSEPbc).

We emphasize that our aim here is to provide gen-eral diagnostics for assessing the performance of anypretreatment method, as opposed to determining anoptimal pretreatment method for any given application.We present this evaluation in the context of examplepretreatment methods applied to three spectral datasets formulated to yield analysis challenges represen-tative of those commonly encountered in practice. Theresults illustrate a range of effects that representativepretreatment methods can have on calibration models.The methodology we propose enables the visualizationof complex multivariate models in terms of simple di-agnostic graphs that can facilitate a wise choice of validcalibration models, offering a means to significantly im-prove the capability of a spectrophotometric analysis.

Theory

Spectral pretreatment methods

Spectrophotometrics often aims to provide an alterna-tive to the slow, expensive measurement of a propertyof interest using stoichiometric calibration techniques.However, prediction obtained from a multivariate cali-bration model is sensitive to spectral interference suchas baseline variations, matrix effects, noise, etc. Manysignal pretreatment strategies have been adopted toreduce such spectral distortions. As surveyed above,pretreatment methods in spectrophotometric analysisfall principally in three categories: (1) geometric spec-tral preprocessing, (2) dimensionality reduction, and(3) combinations of these pretreatment methods. Theprevious literature details these classical methods, andhere we discuss only how to validate the calibrationmodels following signal preprocessing.

One can combine pretreatment methods to deal withdiverse sources of spectral interference in a complexmatrix. In practice, spectral interference can arise frominstrumental and environmental changes, as well asthe presence of intrinsic matrix interference, makingit difficult to untangle characteristic signals with asingle pretreatment approach. No standard procedureexists by which to determine whether pretreatments in

Validity following preprocessing 2319

combination can give an optimal quantification resultfor the purpose of developing a calibration model. Itis therefore very important to establish appropriatecriteria to assess the outcomes of pretreatments appliedindividually or in combination.

In this work, we propose two validity parameters,prediction bias and prediction precision, to evaluate thevalidity of a set of calibration models following spectralpretreatment.

Prediction bias

The presence of spectral interference will usually causea significant bias in prediction [14]. Thus, it is reason-able to believe that an estimate of bias can providea straightforward means to determine the degree towhich a given pretreatment approach succeeds in sup-pressing spectral interference. One can diagnose thepresence of bias in the analysis of a set of standards bylinear regression [29],

y = a + b y (1)

where y and y are the predicted and reference values,respectively.

Ideally, the values predicted by application of a cal-ibration model relate to known quantities by a straightline with a slope of 1 and an intercept of 0. The presenceof systematic and random errors in an analytical proce-dure produces a deviation from this ideal limit [30]. Todetect potential bias, we propose to examine the EJCR,which gauges an acceptable range for the intercept andslope combined.

A simple equation describes the region of jointconfidence [26, 30]:

n(a − a

)2 + 2

(n∑

i=1

ci

)(a − a

) (b − b

)

+(

n∑

i=1

c2i

)(b − b

)2 = 2s2 Fα,2,n−2 (2)

where n is the number of samples, ci is the actual con-centrations, s2 is the regression variance, and Fα,2,n−2

is the critical F value with 2 and (n − 2) degrees offreedom at a given (1 − α)−percent confidence levelis usually 95 %. If, following pretreatment, the point(0, 1) moves inside the EJCR, then we can confidentlyconclude that prediction bias is absent, indicating thatpretreatment has adequately suppressed spectral in-terference. We note that the elliptic area depends onthe sample size. The use of an appropriate method ofsample design, such as ASTM E1655-05, is thereforecritical for the effective implementation of this metric.

The most straightforward way to determine whetherthe point (0, 1) lies in the ellipse is to calculate thetwo elliptic intercepts with the straight line a = 0 [25].We define two intercepts as (0, b 1) and (0, b 2), withb 2 > b 1. When we find the condition for which b 2 >

1 and b 1 < 1 simultaneously, we know that the point(0, 1) lies inside the EJCR. González et al. gives anexpression to calculate these two intercepts [30]. Forconvenience, we reproduce its derivation here.

Setting a = 0 and y = b − b , Eq. 2 becomes asfollows:(

n∑

i=1

c2i

)

y2 + 2a

(n∑

i=1

ci

)

y + na2 − 2s2 Fα,2,n−2 = 0

(3)

In Eq. 3, we let

A =n∑

i=1

c2i (4)

B = 2an∑

i=1

ci (5)

and

C = na2 − 2s2 Fα,2,n−2 (6)

so that the roots for y simply become

−B ± √B2 − 4AC

2A(7)

and, using these two roots, we find the intercepts b 1 andb 2 simply by equating

bi = b − yi. (8)

In this way, the criterion [b 1, b 2] conveniently deter-mines whether the point (0, 1) lies within the EJCR.

Less graphically, the correlation between slope andintercept can be determined from the properties of theregression matrix as

corr(a, b) = −y√

n∑

yi. (9)

The magnitude and sign of the correlation relate the re-gion of joint confidence to the rectangular area definedby the confidence intervals of the slope and interceptdefined separately [31].

Random errors can affect calibration models in waysthat are difficult to predict. It is, thus, often advis-able to filter noise before calibration when a set ofspectra present a low signal-to-noise ratio (SNR). Onecan conveniently improve the SNR of a set of Ramanspectra by using Savitzky–Golay smoothing [20] or


applying wavelet transform denoising techniques [21].The present work focuses on the influence of struc-tured errors, such as instrumental and environmentalvariations, and matrix interference that might appearin raw spectra, or artifacts introduced by inappropriatepretreatment strategies.

Prediction precision

When a calibration result passes the EJCR test, one canassume that the underlying model is reliable. However,the EJCR criterion alone does not suffice to determinethat a particular pretreatment has produced the opti-mum calibration because one often finds that severaldifferent candidate models satisfy the EJCR condition.When validating a calibration model, one should selectfrom among such possibilities by evaluating the corre-sponding prediction precision.

Below we illustrate the problem and present aneffective means to solve it by gauging the predictionprecision with reference to the criteria of bias-correctedroot mean square error of prediction and cross valida-tion (RMSECVbc). Compared with RMSEP, RMSEPbc

greatly reduces the risk of underestimating the calibra-tion performance in the presence of significant bias. Wecan calculate RMSEPbc or RMSECVbc by evaluatingthe following:

RMSEPbc or RMSECVbc =√∑n

i=1 (yi − yi − bias)2

n − 1

(10)

bias =∑n

i=1

(yi − yi

)

n(11)

where yi equals a value obtained either from indepen-dent prediction or cross validation, and yi is the knownor reference value of a sample i. n is the number ofsamples.

Equation 12 yields the bias-corrected relative rootmean square error of prediction, RRMSEPbc.

RRMSEPbc = 100 × RMSEPbc/cn (12)

where cn is the mean of the calibration concentrations,and we express RRMSEPbc as a percent. We can obtainan equivalent measure of the RRMSECVbc by usingRMSECVbc in Eq. 12 in place of RMSEPbc.

Having constructed a calibration model, we applythe EJCR test and calculate the RMSEPbc. If theRMSEPbc is satisfactory and the EJCR test detectsno significant bias, we can conclude that the signal

preprocessing has succeeded and deem the calibrationmodel valid for future application.

Experimental procedure

Raman instrument

We record Raman spectra using a SpectraCode modelRP-1 spectrometer. This system combines a spectro-graph (Acton 150 mm f/4.0) with a thermoelectri-cally cooled CCD detector (Princeton Instruments1,024 × 256, 20 μm pixels) and a backscattering probethat uses a Semrock notched beam splitter with spatialfiltering to provide near-total stray light rejection.

We collect signal recorded by the CCD on a lab-oratory computer operating a LabVIEW data acqui-sition program and process the resulting files off-lineusing multivariate analysis algorithms described above,which we have developed using MATLAB.

Data sets

Data set 1

We recorded Raman spectra of 42 aqueous solutionscontaining two solute components, lactic acid, andmelamine, with concentrations ranging from 0.35 to1.80 g/L and 0.35 to 1.36 g/L, respectively. To mini-mize bias in calibration, we chose the concentrationsof lactic acid and melamine according to the princi-ples of uniform design [32]. Accordingly, our predic-tion bias comes mainly from spectral interference (e.g.,stray light, fluorescence, and CCD read noise). TheRaman scattering cross section of lactic acid is veryweak, whereas that of melamine is very strong. Thischaracteristic of these analytes enables a comparativestudy of the effects of spectral preprocessing on strongand weak signals combined, as commonly encounteredin analytical practice. In a protocol conforming withASTM E1655-05, we set aside 20 prediction samplesto simulate the analysis of a batch of real unknownsamples and used the remaining 22 samples as a trainingset.

Data set 2

We recorded Raman spectra of 72 aqueous mixturesof lactic acid, melamine, and milk. For the milk, weused four different brands of commercially available2 % milk and collected 18 samples from each brand tosimulate biological matrix interference that may be en-countered in practice. Samples consisted of 10-ml milk


mixed with 20-ml aqueous solutions of lactic acid andmelamine with varying concentrations. The concentra-tions of lactic acid and melamine ranged from 0.44 to1.95 g/L and 0.60 to 1.80 g/L, respectively, followingthe principles of uniform design. We divided the datafrom spectral measurements on these samples into twoparts, a training set with 40 samples and a validationset with 32 samples, again in conformance with ASTME1655-05.

Data set 3

We derived a set of solid samples prepared from 26commercial bleached softwood Kraft pulps. Each pulptype was beaten to varying degrees in a PFI mill to yieldfinal samples in each case with five different revolu-tion values. In total, the complete sample set consistedof 130 representative hand sheets for which separatemeasurements established 15 physico-mechanical sheetproperties. For each sheet sample, we collected 15Raman spectra at different positions on each sheetsample with sample rotation during acquisition, andaveraged these to form a final spectrum in each casefor processing. Before calibration, we set aside 30 sheetsamples derived from a randomly selected six of the26 kraft pulps as validation set, thereby minimizingthe bias introduced by sample selection. We used theremaining sheet samples, representing 16 kraft pulps,as a training set.

Results and discussion

Before building calibration models, we performed vari-ous steps of spectral preprocessing to suppress spectralinterference. In this work, we chose to explore onlysome of the most representative preprocessing strate-gies described in the literature survey above, includ-ing MSC, SG-1D, SG-2D, DWT, CWT, OSC, UVE,DWT-OSC, CWT-OSC, DWT-UVE, and CWT-UVE.We optimized the parameters of each preprocessingmethod selected to yield the best result in leave-one-outcross validation. Processing data sets of a size typicalfor an analytical chemistry application limit the degreeto which we can apply validation methods of lowervariance, such as K-fold cross validation. Utilizing theBreusch–Pagan test, we establish an absence of het-eroscedasticity in our model [33], thus confirming a nor-mal distribution of uncorrelated modeling errors. Forthe purposes of illustrating the effect of preprocessingas observed in spectra, we focus on Data Set 1 because,here, pretreatment reveals features for lactic acid andmelamine that are clear and easy to discriminate.

Effects of preprocessing on prediction bias

After having constructed PLS models with pre-processed spectra, we can evaluate the effects of sig-nal preprocessing on validity simply by comparing theperformance of these models. We focus on two funda-mental performance parameters, prediction bias, andprediction precision. Here, for the purposes of demon-stration, we estimate an approximately optimal numberof PLS factors by leave-one-out cross validation. Thesection to follow will present a more accurate methodto achieve this optimization.

Figure 1 illustrates the EJCR obtained for the pre-diction of lactic acid concentration in data set 1 fol-lowing different preprocessing methods. As shown inFig. 1a, a PLS model built with raw spectra presentssignificant bias, indicating that the absorption of waterin these aqueous Raman spectra plays the role of avarying background. MSC does not correct this inter-ference, establishing that the main sources of inter-ference are not multiplicative. SG-1D, SG-2D, DWT,and CWT all succeed in suppressing the interferenceof background and noise, yielding in each case a validcalibration model. It can be seen from Fig. 1b that OSCfails to remove varying baselines, causing a significantbias in prediction. Feature selection with UVE appar-ently reduces the effects of systematic interference onthe calibration model.

As expected, Fig. 1c confirms that the calibra-tion models constructed with DWT-OSC, CWT-OSC, DWT-UVE, and CWT-UVE yield valid re-sults, offering quantitative proof that a combination ofgeometric and dimensionality reduction pretreatmentmethods can represent a good strategy for suppressingcomplex interference.

Figure 2 plots the EJCR associated with the predic-tion of melamine in data set 1 following the applicationof different preprocessing methods. There is no reasonto expect all of the calibration models to rise to thesame level of validity. Thus, the apparent uniformity inEJCR conformance simply indicates that preprocessinghas only a slight effect on strong signals. The resultspresented in Figs. 1 and 2 illustrate how much more aweaker a Raman response can benefit from appropriatesignal preprocessing.

Figure 3 illustrates the effects of representative sig-nal preprocessing methods on elliptical confidence re-gions for data sets 2 and 3. In the interest of space,we will consider only the performance of some rep-resentative preprocessing methods. The following willcompare EJCR analyses of calibration models built onraw spectra with DWT, UVE, and DWT-UVE pre-processing, noting that the variable selection technique


Fig. 1 Elliptic joint confidence region plots obtained followingdifferent preprocessing methods applied to spectra of lactic acidin data set 1. a Geometric pretreatment methods comparingmodels derived from raw spectra (black square), MSC (circle),SG-1D (black triangle), SG-2D (diamond), DWT (asterisk), andCWT(plus sign). b Dimensionality reduction methods comparingmodels derived from spectra processed using OSC (black square)and UVE (circle). c Combinations of pretreatment methods com-paring models derived from the spectra processed using DWT-OSC (black square), CWT-OSC (circle), DWT-UVE (plus sign),and CWT(asterisk)

will often significantly change the results of calibrationmodels compared with that of OSC [18].

As shown in Fig. 3a, these signal preprocessing meth-ods affect calibration models for lactic acid in data set2 quite differently from those for lactic acid in data set1. We can explain this by noting that that the chemical

Fig. 2 Elliptic joint confidence region plots obtained followingdifferent preprocessing methods applied to of melamine in dataset 1. a Geometric pretreatment methods comparing modelsderived from raw spectra (black square), MSC (circle), SG-1D (black triangle), SG-2D (diamond), DWT (asterisk) andCWT(plus sign). b Dimensionality reduction methods comparingmodels derived from spectra processed using OSC (black square)and UVE (circle). c Combinations of pretreatment methods com-paring models derived from spectra processed using DWT-OSC(black square), CWT-OSC (circle), DWT-UVE (plus sign), andCWT(asterisk)

interactions between protein components in milk andlactic acid significantly modify the Raman spectrum ofthis component when compared with that of lactic acidin water. The diverse sources of spectral interferencemake it difficult to suppress their effects with a singlepreprocessing method.

Clearly, b1 and b2 (in Eq. 8) obtained with DWTand UVE exceed 1, indicating that a single signal


Fig. 3 Effects of somerepresentative methods onelliptic joint confidenceregion plots for a lactic acid indata set 2, b melamine in dataset 2, c tensile index in dataset 3, and d TEA in data set 3,comparing in each casemodels derived from rawspectra (black square) andspectra processed using DWT(circle), UVE (plus sign), andDWT-UVE (asterisk)

preprocessing method only suppresses part of the spec-tral interference. The complexity of the matrix in thiscase calls for appropriate combinations of signal pre-treatments. Consistent with observations above, DWT-UVE improves prediction performance, yielding a validcalibration model. The behavior exhibited in Fig. 3bcalls for conclusions similar to those for Fig. 2b.

Turning to the Raman spectra of pulp handsheets,we note that sheet properties, unlike the simple spec-tral signals that vary with concentrations of analytesin solution, arise from a synergistic effect of manyphysical and chemical attributes. Thus, we can expectthat a large proportion of the spectral information,outside of background and noise, will be useful forpredicting sheet properties. Analysis of Raman spectraof sheets confronts the challenge of extensively over-lapped features owing to the presence of hundreds ofcompounds at detectable levels. An inappropriate sig-nal preprocessing may lead to a loss of useful informa-tion. With EJCR, an appropriate signal preprocessingmethod can be optimized to produce a satisfactorycalibration model as indicated in Fig. 3c, d, wherethe DWT-UVE model provides satisfactory predictionperformance.

All the calibration models tested in Figs. 1, 2, and3 serve to show that the EJCR criterion can providea straightforward means to establish the effect of pre-

processing method on calibration model, simplifyingthe task of developing an optimal analysis strategy.

Effects of signal preprocessing methods ondetermination of PLS factors

As discussed in the previous section, an appropriatesignal preprocessing method can effectively removespectral interference and reduce model dimensionalityand, by this, improve the performance of a multivariatecalibration model. In this respect, the pulp handsheetsamples represent a realistically complex sample set.

It is generally thought that smaller models tendto be more robust than larger models since robust-ness is known to degrade with increasing model di-mensionality [34]. Applying the principle of minimumRMSEPbc obtained by leave-one-out cross validationusually causes over-fitting [35, 36]. In order to ex-plore conditions under which we might reduce therisk of over-fitting, we adopt the F test criterion withalpha = 0.05 (95% confidence level) and comparethe MSECVbc of models with different PLS factorsto determine the range of PLS models that are notsignificantly different from that of the model with min-imum MSECVbc. [37] We perform EJCR tests to vali-date these PLS models with each increase in number offactors. From this exercise, we can find the valid model


Number of PLS factors

Number of PLS factors

RM

SE

CV

bcR

MS

EC

Vbc

RawDWTUVEDWT-UVE

RawDWTUVEDWT-UVE

(a)

(b)

Fig. 4 Relationship between PLS factors and RMSECVbc ob-tained with representative preprocessing methods for a tensileindex and b TEA in data set 3, where the factor number markedwith a star is selected

with a minimum number PLS factors and recognize itas a best predictor with a minimum risk of over-fitting.If no model can pass the EJCR test, the number of

PLS factors corresponding to the minimum MSECVbc

is selected as the optimal one. However, this wouldrequire further inspection of the model structure toremove the spectral interference evidently present incalibration model.

Figure 4 shows how the RMSECVbc varies as afunction of the number of PLS factors for differentdata pretreatment methods, where we use a star tomark the ellipse corresponding to the best numberof PLS factors for calibration. Figure 4a, b shows forraw data that the RMSECVbc has not completely set-tled down after extracting more than 20 PLS compo-nents. This means that there is still some systematicinformation related to the analyte left in the data.However, useful information of this kind is usuallyswamped by spectral interference. Including a highnumber of PLS components in the calibration modelusually introduces much more interference than use-ful information, increasing the risk of over-fitting. Asexpected, signal preprocessing methods effectively de-crease the complexity of corresponding PLS models.Most excitingly, we show that an appropriate combi-nation of signal preprocessing, e.g., DWT-UVE, candecrease both a model dimensionality and its predictionerror.

Prediction results

Table 1 summarizes the accuracy with which PLSmodels predict the concentrations of lactic acid andmelamine in data set 1 following various methods ofdata pretreatment. The quality of prediction for lactic

Table 1 Prediction results, including optimal number of PLS factors, bias-corrected RMSEP, correlation coefficient, and indication ofconformance with the EJCR test obtained following various pretreatments for data set 1, aqueous lactic acid, and melamine

Parameter Raw MSC SG-1D SG-2D DWT CWT OSC UVE DWT- CWT- DWT- CWT-OSC OSC UVE UVE

Lactic acidPLS factors 6 6 4 4 4 4 4 4 2 2 3 3RMSEP 0.065 0.056 0.041 0.045 0.042 0.045 0.057 0.078 0.043 0.045 0.045 0.053RMSEPbc 0.061 0.054 0.041 0.044 0.041 0.045 0.055 0.077 0.042 0.044 0.045 0.052RRMSEPbc (%) 6.60 5.87 4.38 4.75 4.32 4.84 5.94 8.30 4.58 4.81 4.85 5.70Q2 0.977 0.981 0.988 0.987 0.985 0.984 0.979 0.977 0.982 0.980 0.985 0.970R 0.994 0.995 0.997 0.996 0.997 0.996 0.997 0.989 0.997 0.996 0.996 0.995EJCR (Y/N) N N Y Y Y Y N Y Y Y Y Y

MelaminePLS factors 5 4 3 3 3 2 3 3 2 2 3 2RMSEP 0.013 0.012 0.007 0.007 0.007 0.008 0.009 0.008 0.007 0.008 0.008 0.008RMSEPbc 0.012 0.011 0.007 0.007 0.007 0.008 0.009 0.008 0.007 0.008 0.008 0.008RRMSEPbc (%) 1.71 1.66 0.97 0.98 0.97 1.18 1.32 1.24 0.96 1.11 1.11 1.20Q2 0.997 0.997 0.998 0.998 0.998 0.998 0.998 0.997 0.999 0.996 0.998 0.999R 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999EJCR (Y/N) Y Y Y Y Y Y Y Y Y Y Y Y


acid differs from that of melamine to some degree.The background produced by solvent water has a greateffect on the calibration of lactic acid, leading to asignificant deviation in its prediction. Suitable signalpreprocessing methods play an essential role in im-proving the performance of the calibration model, andinappropriate signal preprocessing appear to spoil cal-ibration outcomes. EJCR validation of the PLS mod-els during development is seen to greatly improvethe prediction reliability. As for melamine, the strongabsorption bands make the calibration model robustagainst interference, resulting in a reliable calibrationmodel regardless of pretreatment strategy. Even so,the choice of a most appropriate signal preprocessingmethod improves the performance of calibration modeland reduces the dimensionality of the necessary PLSspace.

The results obtained from data set 2 differ somewhatfrom those of data set 1 as indicated in Table 2. Thisreflects the fact that the sources of spectral interferencein the milk system are much more complicated thanthose in the aqueous data set 1. Milk contains hundredsof detectable compounds, and chemical interactions be-tween sample components (e.g., casein and lactic acid)can modify the Raman spectra of a pure component.The weak absorption bands of lactic acid are totallyswamped and overlapped by absorptions of the milkmatrix, making it a challenge to extract quantitativeinformation from the raw spectra.

As shown in Table 2, the lactic acid models presentevidence of a significant bias when constructed us-ing spectra treated with a single signal preprocess-

ing method. The nature of the system thus calls fora combination of signal pretreatments as a meansto suppress its diverse sources of spectral interfer-ence. As expected, appropriate combinations of sig-nal pretreatments, e.g., DWT-OSC, DWT-UVE, andCWT-UVE, effectively suppress the main sources ofinterference, producing a reliable calibration models.However, the RMSEPbc of these three models arenot significantly lower than that of other models withsole signal pretreatments, indicating that the RMSEPbc

criterion alone cannot detect the presence of predic-tion bias. In comparison to lactic acid models, themelamine models are relatively easy to construct, thereason being that the Raman bands of melamine arestrong and differ from the features of matrix. TheCWT-UVE method provides the optimal calibrationmodel.

Table 3 summarizes parameters characterizing thePLS calibration models developed to predict thephysical properties of paper sheet from pulp fiberRaman spectra pretreated as described above. Theseresults lead to conclusions similar to those for Tables1 and 2, expecting the difference between RMSEP andRMSEPbc. In data sets 1 and 2, the RMSEP differs littlefrom RMSEPbc even when significant bias is present,the reason being that all the experimental factors arewell controlled in these two data sets. In contrast,the RMSEP derived from data set 3 severely deviatesfrom RMSEPbc when significant bias is present. Incases like this, RMSEP underestimates the calibrationperformance, which could motivate an inappropriatesignal preprocessing. We can therefore recommend the

Table 2 Prediction results, including optimal number of PLS factors, bias-corrected RMSEP, correlation coefficient, and indication ofconformance with the EJCR test obtained following various pretreatments for data set 2, lactic acid, and melamine in milk


Lactic acidPLS factors 10 6 6 5 5 4 4 5 3 2 4 2RMSEP 0.21 0.18 0.19 0.18 0.15 0.16 0.15 0.25 0.17 0.18 0.12 0.17RMSEPbc 0.19 0.17 0.16 0.15 0.14 0.14 0.12 0.21 0.16 0.16 0.12 0.16RRMSEPbc (%) 18.5 16.6 16.1 15.0 13.9 14.1 12.2 21.2 16.3 16.1 11.8 16.0Q2 0.916 0.942 0.951 0.946 0.947 0.928 0.979 0.950 0.981 0.995 0.967 0.920R 0.925 0.950 0.958 0.965 0.963 0.966 0.967 0.935 0.950 0.960 0.973 0.948EJCR (Y/N) N N N N N N N N Y N Y Y

MelaminePLS factors 8 8 5 5 5 5 5 6 2 2 4 3RMSEP 0.06 0.06 0.05 0.05 0.05 0.05 0.12 0.06 0.06 0.06 0.06 0.05RMSEPbc 0.06 0.05 0.05 0.05 0.05 0.05 0.11 0.06 0.06 0.06 0.06 0.05RRMSEPbc (%) 9.4 8.2 7.6 7.7 7.8 8.1 16.6 8.5 8.6 9.30 8.5 6.9Q2 0.951 0.948 0.974 0.972 0.975 0.969 0.992 0.939 0.998 0.951 0.973 0.968R 0.973 0.962 0.978 0.973 0.978 0.973 0.878 0.973 0.974 0.967 0.974 0.979EJCR (Y/N) Y N Y Y Y Y N Y Y Y Y Y


Table 3 Prediction results, including optimal number of PLSfactors, bias-corrected RMSEP, correlation coefficient, and in-dication of conformance with the EJCR test obtained following

various pretreatments for data set 3, in which we classify thephysical properties of paper sheet from the Raman spectra ofpulp fiber samples


Tensile indexPLS factors 12 8 9 7 7 6 7 8 5 5 5 5RMSEP 11.3 16.9 15.1 10.9 11.3 17.2 15.3 17.6 10.3 9.4 8.8 9.5RMSEPbc 10.6 14.6 10.2 10.1 9.9 13.5 11.7 14.9 9.6 9.2 8.7 9.3RRMSEPbc (%) 13.3 18.5 12.8 12.8 12.5 17.1 14.8 18.8 12.1 11.6 11.0 11.7Q2 0.886 0.851 0.863 0.872 0.861 0.834 0.959 0.606 0.871 0.766 0.862 0.843R 0.950 0.886 0.948 0.945 0.951 0.954 0.910 0.848 0.941 0.943 0.951 0.947EJCR (Y/N) Y N N Y N N N N Y Y Y Y

TEAPLS factors 13 8 8 6 7 6 9 7 5 6 5 5RMSEP 315 381 389 321 327 361 383 375 336 323 258 342RMSEPbc 307 344 317 310 312 329 359 343 329 299 249 324RRMSEPbc (%) 17.7 19.8 18.3 17.8 17.9 18.9 20.7 19.7 19.0 17.2 14.3 18.7Q2 0.866 0.829 0.838 0.842 0.829 0.824 0.783 0.822 0.849 0.785 0.843 0.816R 0.913 0.882 0.917 0.918 0.910 0.918 0.876 0.906 0.893 0.919 0.937 0.899EJCR (Y/N) Y N N Y Y N N N Y N Y N

Results tabulated for Tensile Index and TEA as examples

criterion of RMSEPbc for the validation of models withuncontrolled interference.

The prediction results characterized in all three ta-bles show that appropriate combinations of signal pre-treatments can produce a better and more reliablecalibration model than those of a sole signal pre-treatment, and inappropriate signal pretreatments canworsen calibration results. More generally, assessmentof both of prediction bias and error can give chemistssufficient information to decide whether a signal pre-treatment is effective and then construct a reliable andaccurate calibration model for prediction.

Conclusion

In the present work, we have developed a statistical cri-terion to assess the performance of calibration modelsdeveloped following the application of different signalpreprocessing methods. Our assessment focuses on twobasic performance parameters, prediction bias, and er-ror, as tested by EJCR and RMSEPbc, respectively.

By means of analyzing a very large range of datasets and pretreatment strategies in combination, weestablish that the EJCR criterion can both measurethe effectiveness of a signal pretreatment on the sup-pression of spectral interference, and indicate a mostreasonable level of model complexity.

Combining the EJCR test with an assessment ofRMSEPbc, we propose criteria that enable a moreconfident choice of best calibration model. A good

model should possess three properties simultaneously:low prediction bias, low prediction error, and lowmodel dimensionality. The criteria proposed here pro-vide a comprehensive set of diagnostics for datapretreatment, orienting a pretreatment strategy mostcorrectly toward a suppression of undesired spectralinterference.

In general, we find that an appropriate combinationof geometric and dimensionality reduction methods canperform better than a single data pretreatment method,serving more effectively to enhance the performance ofa calibration model, and reduce the risk of over-fitting.

The approach described here should apply gener-ally to a broad range of spectral methods, as well asother methods using multivariate analysis. These re-sults should aid analysts seeking to optimize calibrationmodels based on different signal pretreatment methodsand, thus, facilitate the application of spectrophotomet-ric analysis

Acknowledgements This work was supported by the NaturalSciences and Engineering Research Council of Canada, theCanada Foundation for Innovation, and the British ColumbiaKnowledge Development Fund. Da Chen gratefully acknowl-edges the support from the 111 Project (no. B07014) and theInnovation Foundation of Tianjin University (no. 60302048).

References

1. Webb-Robertson B, Bailey V, Fansler S, Wilkins M, Hess N(2012) Anal Bioanal Chem 404:563


2. Rudigier M, Haubner R (2012) Anal Bioanal Chem 403:6753. Cletus B, Olds W, Izake E, Sundarajoo S, Fredericks P,

Jaatinen E (2012) Anal Bioanal Chem 403:2554. Daszykowski M, Wrobel M, Czarnik-Matusewicz H,

Walczak B (2008) Analyst 133:15235. Shao X, Du G, Jing M, Cai W (2012) Chemometr Intell Lab

Syst 114:446. Shetty N, Rinnan Å, Gislum R (2011) Chemometr Intell Lab

Syst 111:597. Lavine B, Workman J (2010) Anal Chem 82:46998. Workman J, Koch M, Lavine B, Chrisman R (2009) Anal

Chem 81:46239. Olivieri AC, Faber NM, Ferré J, Boque R, Kalivas JH,

Mark H (2006) Pure Appl Chem 78:63310. Xu QS, Liang YZ, Shen HL (2001) J Chemom 15:13511. Goicoechea HC, Olivieri AC (2001) Analyst 126:110512. Wold S, Martens H, Wold H (1983) Lecture notes in mathe-

matics. Springer, Heidelberg13. Ding Q, Small GW, Arnold MA (1999) Appl Spectrosc

53:40214. Miller JC, Miller JN (1993) Statistics for analytical chemistry,

3rd edn. Prentice Hall, Chechister, UK15. Ni WD, Brown SD, Man RL (2009) Anal Chem 81:896216. Lavine B, Workman J (2008) Anal Chem 80:451917. Gabrielsson J, Trygg J (2006) Crit Rev Anal Chem 36:24318. Zeaiter M, Roger JM, Bellon-Maurel V (2005) Trends Anal

Chem 24:43719. Naes T, Isaksson T, Kowalski B (1990) Anal Chem 62:664

20. Gorry PA (1990) Anal Chem 62:57021. Chen D, Hu B, Shao XG, Su QD (2004) Anal Bioanal Chem

379:14322. Chen D, Wang F, Shao XC, Su QD (2003) Analyst 128:120023. Centner V, Massart DL, de Noord OE, de Jong S,

Vandeginste BM, Sterna C (1996) Anal Chem 68:385124. Faber NM (2000) J Chemom 14:36325. Liu SM, Kokot S, Will G (2009) J Photochem Photobio C

10:15926. Rius A, Callao MP, Ferré J, Rius FX (1997) Anal Chim Acta

337:28727. Riu J, Rius FX (1996) Anal Chem 68:185128. Pereira AMT, Brandao PRG (2001) Miner Eng 14:165929. González AG, Asuero AG, Fresenius A (1993) Anal Chem

346:88530. González AG, Herrador MA, Asuero AG (1999) Talanta

48:72931. Draper N, Smith H (1981) Applied regression analysis, 766.

Wiley, New York32. Liang YZ, Fang KT, Xu QS (2001) Chemom Intell Lab Syst

58:4333. Montes-Rojas G, Sosa-Escudero W (2011) J Econom

160(2):30034. Faber NM, Duewer DL, Choquette SJ, Green TL,

Chesler SN, (1998) Anal. Chem. 70:297235. Faber NM, Rajkó R (2007) Anal Chim Acta 595:9836. Xu QS, Liang YZ, (2001) Chemom Intell Lab Syst 56:137. Haaland DM, Thomas EV (1988) Anal Chem 60:1193

evaluating the validity of spectral calibration models for quantitative analysis following signal...

Documents