deliverable d.4.04 establishing hyperspectral measurement ... · as described in d4.01, current...

36
SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol Co-founded by the European Commission Page 1 of 36 Deliverable D.4.04 Establishing hyperspectral measurement protocol WP4 – Multi-sensor model-based quality of mountain forest production Task 4.3 – Evaluation of hyperspectral imaging (HI) for the determination of log/biomass HI quality index Revision: Final Author(s): Andreas Zitek, Katharina Böhm, Ferenc Firtha, Barbara Hinterstoisser Partners: BOKU Dissemination level PU (Public) Contributor(s) (BOKU), Jakub Sandak (CNR) Reviewer(s) Anna Sandak (CNR), Federico Prandi (GRAPHITECH) Editor(s) Raffaele De Amicis (GRAPHITECH) Partner in charge(s) BOKU Due date 31-Nov-14 Submission Date 04-May-15

Upload: others

Post on 14-Sep-2019

1 views

Category:

Documents


0 download

TRANSCRIPT

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 1 of 36

Deliverable D.4.04

Establishing hyperspectral measurement protocol WP4 – Multi-sensor model-based quality of mountain forest production Task 4.3 – Evaluation of hyperspectral imaging (HI) for the determination of log/biomass HI quality index Revision: Final

Author(s): Andreas Zitek, Katharina Böhm, Ferenc Firtha, Barbara Hinterstoisser

Partners: BOKU

Dissemination level PU (Public)

Contributor(s) (BOKU), Jakub Sandak (CNR)

Reviewer(s) Anna Sandak (CNR), Federico Prandi (GRAPHITECH)

Editor(s) Raffaele De Amicis (GRAPHITECH)

Partner in charge(s) BOKU

Due date 31-Nov-14

Submission Date 04-May-15

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 2 of 36

REVISION HISTORY AND STATEMENT OF ORIGINALITY

Revision History

Revision Date Author Organisation Description

1.0 01.11.2015 Andreas Zitek BOKU First draft

2.0 15.01.2015 Katharina Böhm BOKU Revision

3.0 08.03.2015 Andreas Zitek BOKU Revision

4.0 19.03.2015 Anna Sandak CNR Revision

5.0 1.04.2015 Jakub Sandak CNR Revision

6.0 2.04.2015 Andreas Zitek BOKU Partner contributions integration

7.0 4.05.2015 Andreas Zitek BOKU Final draft

Statement of originality

This deliverable contains original unpublished work except where clearly indicated otherwise. Acknowledgement of previously published material and of the work of others has been made through appropriate citation, quotation or both.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 3 of 36

Table of contents 1 Abstract and problem statement ......................................................................... 7

2 Introduction .......................................................................................................... 8

2.1 What is NIR and what is hyperspectral imaging (HSI)? .................................... 8 2.2 Setup of HSI systems ........................................................................................ 9

2.2.1 General setup ......................................................................................... 9 2.2.2 Sensor types ........................................................................................... 9 2.2.3 Spatial resolution.................................................................................. 10 2.2.4 Data gathered by HSI ............................................................................ 10 2.2.5 Analysis of HSI data – workflow ........................................................... 12

3 Potential applications of HI to wood samples within SLOPE scenario ............... 17

4 Protocol for testing of the hyperspectral imaging along the whole process chain within SLOPE scenario (Task 4.3) ........................................................................ 18

4.1 Forest modeling and inventory ...................................................................... 19 4.2 Tree marking and cutting of tree ................................................................... 19 4.3 Processor head ............................................................................................... 19

4.3.1 Potential instrumentation and sensors ................................................ 20 4.4 Pile of logs ...................................................................................................... 21 4.5 Laboratory ...................................................................................................... 21

4.5.1 NIR hyperspectral imaging system at BOKU ........................................ 21 4.5.2 VIS-NIR hyperspectral imaging system at CNR ..................................... 24 4.5.3 Sampling of trees for laboratory trials ................................................. 26 4.5.4 Surface conditions and detection performance ................................... 26 4.5.5 Combination of hyperspectral imaging and NIR spectroscopy ............ 27 4.5.6 Wood defects detection and HI quality index ...................................... 27

5 Sensors for the prototype .................................................................................. 28

6 . APPENDIX .......................................................................................................... 29

6.1 Application of Hyperspectral Imaging in different fields ............................... 29 6.1.1 Remote sensing .................................................................................... 29 6.1.2 Environment ......................................................................................... 29 6.1.3 Soil ........................................................................................................ 29 6.1.4 Product and process control ................................................................ 30 6.1.5 Application in food quality and safety control ..................................... 30 6.1.6 Application in archaeology ................................................................... 30 6.1.7 Application in agriculture ..................................................................... 30 6.1.8 Application in plant science ................................................................. 30 6.1.9 In pharmaceutics .................................................................................. 30 6.1.10 Medical applications ............................................................................ 30 6.1.11 Forensic applications ............................................................................ 31

7 References .......................................................................................................... 32

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 4 of 36

List of figures Figure 1. Electromagnetic spectrum (from D 4.03) ................................................... 8

Figure 2. RGB image yielding three greyscale pictures ........................................... 11

Figure 3. SpectroCam™ Multispectral Imaging (MSI) Camera yielding up to 8 greyscale pictures ............................................................................................ 11

Figure 4. HSI system yielding > 100 greyscale pictures (“hypercube”); in a hypercube, each pixel carries the spectral information from all measured wavelengths. ................................................................................................... 12

Figure 5. The hypercube with x-y directions and wavelength bands λ, and mean spectra in selected regions of interest. ........................................................... 12

Figure 6. Steps, factors and tools to be considered during the hyperspectral imaging process (Duchesne et al. 2012, Vidal & Amigo 2012, Amigo et al. 2013, Huang et al. 2014). ................................................................................ 16

Figure 7. Collection of hyperspectral information and flow of samples/data at different stages of the harvesting process chain according to SLOPE. ........... 18

Figure 8. Experimental set-up for NIR hyperspectral imaging of wood samples .... 22

Figure 9. Ideal indirect illumination for automated inline control by indirect reflected light to minimize surface reflections and spectral artefacts (after Boldrini et al. 2012). ........................................................................................ 23

Figure 10. Experimental set-up for VIS-NIR hyperspectral imaging of wood samples ......................................................................................................................... 25

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 5 of 36

Acronyms ANN Artificial Neural Networks BLS Baseline Shift CCD (silicon based) charge-coupled device

CMOS complementary metal oxide semiconductor HgCdTe Mercury Cadmium Telluride (detector)

HSI Hyperspectral Imaging InGaAs Indium Gallium Arsenide (detector)

LDA Linear Discriminant Analysis MIA Multivariate Image Analysis MLR Multi-linear Regression MLS Modified Partial Least Squares Regression NIR Near Infrared PCA Principal Component Analysis PCR Principal Component Regression PLS Partial Least Square Regression

PLS-DA Partial Least Squares Discriminant Analysis RSQ Squared Coefficient of Correlation ROI Region of Interest SEC Standard Error of Calibration

SECV Standard Error of Cross Validation SEP Standard Error of Prediction SID Spectral Information and Divergence

SIMCA Soft Independent Modelling of Class Analogy SNV Standard Normal Variate SVM Support Vector Machines UAV Unpiloted Aerial Vehicle VIS Visible light

VNIR Visible and Near Infrared

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 6 of 36

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 7 of 36

1 Abstract and problem statement

As described in D4.01, current methods for log grading are mainly based on visual rating, with several drawbacks, like operator-dependency, time-consumption and lack of precision. By the combination of non-contact visual methods like NIR measurements and the hyperspectral imaging (HSI) in the visible and near infrared combined in the SLOPE project for the first time under field conditions, an on-line log quality judgment early in the production chain is targeted. The overall goal of T4.3 is the development of a statistical HI quality index of log quality. Both methods (NIR and HSI) are able to obtain several wood characteristics simultaneously, which makes their applicability to detect different types of wood defects highly probable. Especially the combination of the local measurement of specific NIR spectra of different wood defects on samples with the capability of HSI for gathering spatially resolved data with hundreds of spectra per image of a sample is a promising approach. Chemometric methods applied together with multivariate image analysis will be used to explore possibilities to develop an HI index of log quality. Special questions that have to be addressed for a successful potential application of HI in the field are: harsh conditions (temperature, shocks etc.), lightning, surface roughness, contamination of the sample with soil, oil etc., water content (also due to environmental effects like air moisture, rain, snow and ice), distance from the sample and spatial dimension of the log.

This report contains a protocol for the collection and analysis of hyperspectral data within SLOPE project. It briefly introduces the background of the HSI technology including common hardware setups, and describes case studies applying HSI to wood samples. Then the methodological setup for the acquisition of HSI data within the SLOPE project is described, including a short description of the data processing procedures and model creation, and suggestions for technology transfer for field application.

Finally it shows how the results of this task are integrated with the other methods applied in the SLOPE project to obtain an integrated quality index and to sort logs as early as possible in the production chain.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 8 of 36

2 Introduction

2.1 What is NIR and what is hyperspectral imaging (HSI)?

Near infrared (NIR) is part of the electromagnetic spectrum that ranges from radio waves to gamma rays (Figure 1). NIR covers wavelengths from 780 nm and 3 μm. This part of the spectrum is not visible to human eye which is able to “see” light in the range of 390-700 nm.

Figure 1. Electromagnetic spectrum (from D 4.03)

Hyperspectral imaging (HSI) is the combination of imaging and spectroscopy. Spectroscopic analysis can deal with the entire light spectrum from visible to infrared, and light intensity is captured as a function of wavelength. While spectroscopy is the analysis of light being emitted by or transmitted/reflected from materials with regard to its energy in relation to the wavelength from a specified relatively small sampling area on a sample, hyperspectral imaging (HSI) is a technology that is able to attain both spatial and spectral information from an object. Although HSI has been developed for applications in remote sensing, it has become a powerful non-destructive technique for gaining spatially resolved information on the composition of different kinds of materials in different fields of science (see chapter 6 Appendix). Hyperspectral imaging following the definition of Grahn and Geladi (2007) is the spatially resolved spectral imaging w analyzed wavelengths of > 10.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 9 of 36

2.2 Setup of HSI systems

2.2.1 General setup Generally different spectral imaging techniques can be classified based on the wavelength ranges they cover or, more generally, into mapping and imaging techniques (Boldrini et al. 2012). Wavelengths typically used for HSI are in the visible (VIS) range (380-800 nm), the visible and near infrared (VNIR) range (400-1000 nm) or in the near infrared NIR (900-1700 nm). For HSI imaging typically the terminology used in remote sensing is used (Boldrini et al. 2012), in analogy these techniques can be classified into staredown, whishkbroom and pushbroom imaging.

• staring (staredown) imaging (staring imaging is done by a two-dimensional camera capturing the spectral information in each pixel x-, y-plane at once)

• whiskbroom imaging (during whiskbroom imaging the sample is scanned pixel per pixel in the x–y–spatial direction in a sequential manner)

• pushbroom imaging (pushbroom imaging as a line scanning system acquires the information for each pixel in the line at once)

A hyperspectral imaging system is composed by its hard- and software, the most important hardware parts are (Huang et al. 2014):

• a hyper-spectrograph that disperses the wavelengths of the reflected, transmitted, or scattered light and delivers signals to the photosensitive surface of the detector;

• a detector with a camera which obtains both spectral and spatial information simultaneously;

• an objective lens to adjust the range of light acquisition;

• light source to provide illumination, typically halogen-tungsten lamps;

• an objective table fixed to a conveyer belt to hold and transport the sample and finally

• a computer to compose and store the three-dimensional hypercube.

2.2.2 Sensor types Basically, three types of sensors/cameras are in use (Huang et al. 2014):

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 10 of 36

• silicon (Si)-based charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) cameras

• indium gallium arsenide (InGaAs)-based array detectors,

• and mercury cadmium telluride (HgCdTe)-based array detectors.

Working in the range of 300–1100 nm, in food quality and safety analysis CCD cameras as VIS/NIR detectors are most often used, having the advantage of relatively low cost and possibly a wider availability; however to gain good signals at higher wavelengths significant lightning is needed (Huang et al. 2014).

InGaAs array detectors can cover wavelengths from 900–1700 nm, 1000–2200 nm, and 1200–2500 nm, and the HgCdTe array detectors wavelengths from 1000–2600 nm. InGaAs cameras are known to have sensitivity than Si-based cameras in the NIR region, which is especially true for wavelengths above 900 nm (Huang et al. 2014).

2.2.3 Spatial resolution The central element for an imaging system is the choice of the pixel resolution of the detector in combination with the optical resolution of the spectrograph (Boldrini et al. 2012). In principle it makes sense, to select a system with a proper spatial resolution should according to the size of the analyzed objects (Huang et al. 2014). The spatial resolution of the image (in µm) depends on the pixel resolution of the camera (Huang et al. 2014). Pixel resolution of the camera is defined by the wavelength range used and the number of pixels provided in that wavelength domain (Boldrini et al. 2012). As a rule of thumb (Nyquist-Shannon criterion), the number of pixels should be at least three times the figure of the optical resolution (for spatial x- and y-axes of the system) (Boldrini et al. 2012). Again, the number of pixels in the x- and y-axes should be at least three times the size of the object to ensure accurate reproduction (Boldrini et al. 2012). Generally, the spatial resolution can be easily calculated by dividing the scanned spatial distance by the pixel numbers in each image.

2.2.4 Data gathered by HSI By HSI hundreds of spatially resolved spectroscopic bands in each pixel of an image can be gathered forming the so called “hypercube”, a layered representation of all spectral bands in the set of pixels captured during the imaging process.

While a classical RGB image has 3 bands (red, green and blue) yielding 3 greyscale pictures, multispectral images can produce 4-10 grey scale images, hyperspectral imaging is able to capture quasi-continuous information at > 100 wavelengths

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 11 of 36

(Burger & Kaušakytė 2013). An example of the RGB, multispectral and hyperspectral images are presented in Figures 2, 3 and 4 respectively.

These layered spectroscopic signatures at a given location containing up to hundreds of layers of spectra are typically analyzed by statistical procedures like principal component analysis (PCA) or partial least square regression (PLS) to identify specific patterns in the data that can be then linked to different compounds or structural anomalies. Different chemical compounds of the analyzed entities can be characterized by specific reflection and absorbance properties representing typical “spectroscopic signatures”. Multivariate Image Analysis (MIA) applied to the resulting spatial representations of the signatures allows detection and spatial interpretation of defined features. Information can also be selectively derived from selected regions of interest (ROIs) for further analysis.

Hyperspectral imaging acquires hundreds of spectroscopic bands along very narrow steps (e.g. 10 nm) within a defined range yielding a quasi-continuous representation of the spectroscopic information available from one pixel (Figure 5). Whereas, “multispectral imaging” is focusing only on a set of selected relatively small number of non-continuous wavelengths. It is typically achieved by a using set of band-pass optical filters.

Figure 2. RGB image yielding three greyscale pictures

Figure 3. SpectroCam™ Multispectral Imaging (MSI) Camera1 yielding up to 8 greyscale pictures

1 http://www.pixelteq.com/product/spectral-cameras/

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 12 of 36

Figure 4. HSI system yielding > 100 greyscale pictures (“hypercube”); in a hypercube, each pixel carries the spectral information from all measured wavelengths.

Figure 5. The hypercube with x-y directions and wavelength bands λ, and mean spectra in selected regions of interest.

2.2.5 Analysis of HSI data – workflow During the hyperspectral imaging process a huge amount of data are collected with each measured sample (Boldrini et al. 2012). For data reduction and statistical analysis chemometrical methods need to be applied in the field of the hyperspectral image analysis (Geladi et al. 2004, Geladi et al. 2007, Grahn & Geladi 2007, ElMasry & Sun 2010).

Already many studies explicitly describe the chemometric approaches taken for hyperspecral data analysis (Firtha et al. 2008, Burger & Gowen 2011, Dale et al. 2012, Fernández Pierna et al. 2012, Amigo et al. 2013, Burger & Kaušakytė 2013), including pre-processing of hyperspectral images as essential steps before image analysis (Vidal & Amigo 2012) and multivariate image analysis (Lied et al. 2000, Lied & Esbensen 2001, Prats-Montalbán et al. 2011) in detail. There is also a R-script available for the chemometric analysis of spectroscopic data (R: HyperSpec. R package version, Beleites (2014)). Also some authors (Geladi et al. 2004, Duncker & Spiecker 2009, Thumm et al. 2010, Agresti et al. 2013, Fernandes et al. 2013b) provided selected chemometric approaches to the detection of deficits in wood samples by hyperspectral imaging. These sources represent the basis for the

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 13 of 36

development and application of the hyperspectral imaging technology within the SLOPE project.

In short, the following steps are necessary for data collection by hyperspectral imaging and subsequent processing of the hypercube (summarized also in Figure 6):

1. Reflectance calibration

2. Spectral calibration

3. Collection of data from the sample yielding the spectral hypercube.

4. Dead pixel correction and spikes removal, outliers

a. Dead pixels are usually caused by anomalies in the detectors (Vidal & Amigo 2012)

b. Spikes are sudden and sharp rise followed by a sharp decline in the measured spectrum and appear due to an abnormal behavior of the detector, imperfections of electronic circuits or environmental conditions. They can mask details of the image and can lead to miss-identification of the signal of interest (Vidal & Amigo 2012). One of the most common spike detection methods is bymanual supervision (Vidal & Amigo 2012).

5. Background definition and selection of regions of interest can be selected to narrow down the analysis to specific regions on the sample (Vidal & Amigo 2012).

6. Spectral pre-processing

a. By spectral pre-processing the effects of undesirable interferences which affect the spectral measurement (light scattering, particle-size effects or morphological differences, such as surface roughness and detector artefacts) are minimized (Vidal & Amigo 2012, Amigo et al. 2013). A comprehensive description is given by (Dale et al. 2012), and is partially repeated here: the most important methods are polynomial baseline correction, Savitzky - Golay derivative, Standard Normal Variate (SNV), mean-centering and unit variance normalization (Gowen et al. 2007, ElMasry & Sun 2010). SNV e.g. transformation removes the slope variation from spectra caused by scatter and variation of particle size (Candolfi et al. 1999), by derivative conversion irrelevant baseline signals from samples are removed by taking the derivative of the measured responses with respect to the variable

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 14 of 36

number (index) or other relevant axis scale (wavelength, wavenumbers, etc.) into account (Wise et al. 2006). The first derivative is usually used to remove the offset from the sample and de-regulating lower-frequency signals (Wise et al. 2006) while the second derivative will accentuate the higher-frequency enhancing selectivity (Wise et al. 2006). Major achievements are:

i. De-noising

ii. Scatter correction

iii. Normalization

iv. Derivative creation

7. Compression, data reduction and training

a. For this step multivariate analytical tools can be used, e.g. principal component analysis (PCA), principal component regression (PCR), multi-linear regression (MLR), partial least squares regression (PLS), modified partial least squares regression (MPLS), partial least squares discriminant analysis (PLS-DA), linear discriminant analysis (LDA), support vector machines (SVM), artificial neural networks (ANN), soft independent modelling of class analogy (SIMCA) baseline shift (BLS), spectral information and divergence (SID) (Dale et al. 2012). Typical mathematical algorithms applied are (Dale et al. 2012):

i. PCA: used for data compression and information extraction;

ii. MLR: allowed to establish a link between a reduced number of wavelengths (or wavenumber) and a property of the samples and to find a single factor that best correlates predictor (X) variables with predicted (Y) ones;

iii. PLS: used to establish a linear link between two matrices, the spectral data X and the reference values Y; in other words, it attempted to find factors for both capture variance and also to achieve correlation,

b. Unsupervised PCA (principal component analysis) is a multivariate statistical tool used to obtain a simplified representation of correlated multivariate data (Amigo et al. 2013). The orthogonal transformation of data by PCA results in fewer independent

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 15 of 36

variables but maximum representation of original variables (Vidal & Amigo 2012, Zhang et al. 2012).

c. Partial least square discriminant analysis (PLS-DA) can also be used to classify and detect specific characteristics on samples (Lee et al. 2014)

d. Supervised PCA: Classification of pixels that can be agglomerated to features is usually done by applying a statistical classification method (e.g. LDA or PLS-DA) to a subset of spectral data from the hyperspectral image reflecting the target feature. Various methods are available for the creation of calibration and validation datasets based on selected spectra from single or multiple hyperspectral images. However, many supervised methods involve some work on the user’s behalf, typically in selecting regions of interest (ROIs) from imaging data that describe a particular object or material. Spectra selected from ROIs receive a categorical variable according to their class membership, followed by a pooling in a two-dimensional matrix which forms the basis for the development of a classification model. This model is then used to classify the individual spectra contained in the hyperspectral image which then yields a “classification map”. The extent of different features on the map can then be evaluated by counting the pixels in each class (Amigo et al. 2013).

e. Either within the picture or the associated PCA plots data can be selected and used for training for the specific wood characteristics to be analyzed.

8. Feature extraction, image

a. Linear discriminant analysis (LDA) and support vector machine (SVM) can be then used to classify morphological features (Lee et al. 2014). ANN can also be applied for pattern recognition, classification or clustering and quantitative modelling (Dale et al. 2012).

b. Also single wavebands can be selected that might be able to clearly display certain deficits.

9. Results are characteristic wavelength information that can be used for classification and feature selection, yielding the HI quality index.

10. Application of the HI-index for the detection of wood defects.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 16 of 36

11. Assessment of calibration performance: NIR calibration models performances can be characterized by several parameters: standard error of calibration (SEC) or standard error of cross validation (SECV). To perform calibration model performance, an independent set of samples is used to get the standard error of prediction (SEP) and the squared coefficient of correlation (RSQ), which are used to describe the NIR analytical error when analyzing samples of unknown quantitative composition (Dale et al. 2012).

Figure 6. Steps, factors and tools to be considered during the hyperspectral imaging process (Duchesne et al. 2012, Vidal & Amigo 2012, Amigo et al. 2013,

Huang et al. 2014).

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 17 of 36

3 Potential applications of HI to wood samples

within SLOPE scenario

Hyperspectral Imaging has been already applied to wood samples for different purposes in the past. Fernandes et al. (2013a) and Fernandes et al. (2013b) describe the successful measurement of intra-ring wood density by means of VIS/NIR hyperspectral imaging. Geladi et al. (2004) provide an interesting example of the detection of different wood characteristics and deficits (species, living bark, dead bark, living wood, dead wood, cracks, reaction wood) by hyperspectral imaging and chemometric methods. How quick the surface of a freshly cut poplar wood may degrade due to light irradiation changing its hyperspectral signature was shown by Agresti et al. (2013).

The “Detection and classification of Norway spruce compression wood in reflected light by means of hyperspectral image analysis” was described by Duncker and Spiecker (2009). Thumm et al. (2010) mapped the chemical properties of compression wood by spatial NIR measurements. Bharati et al. (2003) described a method how typical wood deficits (like knots, splits, pitch and bark pockets) can be analyzed in a wood production process only by colour and contrast using RGB images using PCA and multivariate image analysis (MIA), with a success rate of 85 %.

The ability to visualize the distribution of wood properties, especially of defects deficits, is the outstanding feature of HSI. However, an application of this technology under field conditions still represents a great challenge and has been only rarely tested under such conditions for the targeted purpose so far.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 18 of 36

4 Protocol for testing of the hyperspectral

imaging along the whole process chain within

SLOPE scenario

The applicability of HSI both in laboratory and field will be extensively tested within Task 4.3 of the SLOPE project. The flowchart presented in Figure 7 summarizes all foreseen activities, sample collection moments, measurement schedule as well as flow of data and quality indicators. The following sub-chapters are providing further details and explanations of actions foreseen on each phase of the SLOPE process chain.

hyperspectral measurement (wet & rough state at differ-

ent temperatures)

compute wet wood HSI quality index#3

cut pieces for drying, wood moisture determination

chemometric models for wet & rough wood and/or in f ield

chemometric models for wet & rough wood (lab)

collect samples: wood logs

measurement hyperspectral image

measurement of hyperspectral imaging

handheld device

compute HSI quality index#2

compute HSI quality index#5

(optional) measurement hyperspectral

image handheld device

compute HSI quality index#6

tree marking

cutting tree

processor head

pile of logs

expert system & data base

condition rough samples to norm climate (20 °C, 60 %)

hyperspectral measurement (cond. grinded state)

compute the log quality class (optimize cross-cut)

estimated tree quality

forest models

update the forest database

compare results of different temperatures, roughness,

wet and dry states

combine all available char-acteristics of the log

lab

calibration transfer f (MC, surface_quality)

3D tree quality index

NIR quality index

stress wave SW quality index

cutting force CF quality index

compute HSI quality index#1

grind samples

Storage of samples in lab (f rozen -20°C)

measure surface roughness & temp

hyperspectral measurement (cond. rough state)

compute dry wood HSI quality index#4

Figure 7. Collection of hyperspectral information and flow of samples/data at different stages of the harvesting process chain according to SLOPE.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 19 of 36

4.1 Forest modeling and inventory

The hyperspectral imaging is a very promising technology when implemented for remote sensing. It can be therefore used within SLOPE project for providing refined information before harvesting. The possibility of installing of the hyperspectral camera on the UAV while surveying the pilots will be investigated in collaboration with SLOPE partners and hardware providers.

Several challenges are expected while analyzing air-born hyperspectral images, such as complex (natural) illumination, special distortions and imperfection in the mosaic reconstruction. However available software tools as well as literature references will be used for refining the processing algorithms and interpretation of results.

4.2 Tree marking and cutting of tree

It is possible to collect hyperspectral images in field during tree marking and/or tree felling. However, this may not provide really meaningful information as no any internal wood defects may be detected. However some preliminary field tests with portable hyper/multispectral camera are foreseen before SLOPE demonstration in order to collect reference data at cutting the tree.

Furthermore, during the tree cutting, a set of tree logs showing different degrees of defects for the development of the hyperspectral quality index in the laboratory will be collected. In the lab the samples will be stored and treated and analysed along a specified methodological pathway to gain the spectra information relevant for each type of deficit under varying environmental conditions.

4.3 Processor head

The implementation of the hyperspectral technology directly on the forest working machines is a main challenge of WP3 and WP4 of the SLOPE project. The optimal set-up for such a system will be designed and scrutinized within task T4.3. However, it will not be a fully functional camera as it cannot be fully protected against the harsh working conditions. Moreover, the working distance of the standard hyperspectral system does not permit a safe system installation. The solution to be explored within SLOPE is to use the high-performing hyperspectral system available at the laboratory and to identify light wavelengths containing most of the meaningful information describing presence and quantity of wood defects. The monochromatic (or multispectral) system will be then installed on

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 20 of 36

the scanning arm of the processor head capable for imaging that specific set of spectral information.

4.3.1 Potential instrumentation and sensors For this deliverable a comprehensive review of the currently available instrumentation with regard to their potential application in the field was conducted. The final solution to be implemented will base on the performance of the spectral information and sensor’s capacity of wood defects identification. The following options were considered when identifying sensor configuration:

1) either one of the existing new HI sensors (IMEC sensor, GAMAYA SA) that can be used for different imaging strategies

2) one of the existing handheld solutions (OCI, etc.)

3) multispectral imaging by a filter system mounted in front of conventional cameras

4) even spatially resolved analysis of single spectra

It has to be mentioned that solutions 3 and 4 are just an option; if the HI quality index can base only on a reduced number of wavelengths (1-10 specific wavelengths). Even if spatially resolved information on a reduced number of wavelengths could be derived from the sample, it strictly speaking does not represent hyperspectral imaging following the definition of Grahn and Geladi (2007), where hyperspectral imaging starts at a number of analyzed wavelengths of > 10.

4.3.1.1 Hyperspectral imaging – whole spectral range measured by hyperspectral camera

Portable HI sensors integrated with the processor head will be used here for the spectra acquisition. HI sensor(s) will be installed on the scanning bar, where they will be protected from the dust and dirt. However, which imaging strategy will be applied will be determined within the next months of the project.

4.3.1.2 Hyperspectral imaging – whole spectral range measured with limited spatial resolution

The alternative solution to the high-cost and low-mechanical resistance camera system is to use a micro spectrometers considered as a single pixel of the line scan camera. Each such sensor measures the whole spectra (in VIS or VIS-NIR ranges) but with limited spatial resolution. It is due to the fact that the external size of the sensor is allowing placing with an effective distance of ~10mm. Various configurations will be tested on the laboratory scanner within T4.3 of the SLOPE project in order to identify the best performing sensor. Whenever preliminary

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 21 of 36

results will allow further development, a series of micro spectrometers will be installed on the scanning arm of the prototyped processor head.

4.3.1.3 Multispectral imaging - selected wavelengths with due filter solutions Multispectral imaging systems could be either based on tunable filters (Gat 2000) or filter wheels (Geladi et al. 2004). However, the acquisition of pictures might take some time, potentially limiting their application within the project.

4.3.1.4 Mono-spectral imaging - selected wavelength(s) custom line scan camera

The ultimate solution for implementation of the hyperspectral imaging toward tree harvesting process is to explore this technique within the laboratory testing campaign. The output of such an activity would be a list of spectral bands mostly related to the selected wood defects. The custom sensor will be developed on this base and will consist of the linear array of photodiodes/detectors. The light wavelength of interest will be emitted by the light source, preferably LED (light emitting photodiode). Such solution will be most cost-effective and also minimizes risk of the sensitive equipment damage when implemented on the machine working in harsh environment such as forest.

4.4 Pile of logs

In analogy to the tree marking, it is possible to acquire hyperspectral images from logs stored in plies before removing from the forest. Again, the quantity of information to be gained as well as difficulties with the specific logs identification does not encourage significant research efforts at this processing stage. Only sample hyper/multi spectral images will be collected for the illustration purposes.

4.5 Laboratory

The most intensive research campaign as related to the hyperspectral imaging is foreseen to be conducted at the laboratories of BOKU and CNR. The first will focus on testing the hyperspectral NIR system, when the second lab will investigate capabilities of hyperspectral system in the VIS-NIR range. Some more details regarding the soft- and hardware solutions as well as list of foreseen challenges are briefly summarized below:

4.5.1 NIR hyperspectral imaging system at BOKU

4.5.1.1 NIR hyperspectral camera The hyperspectral experimental setup available at BOKU University is shown in Figure 8 and is similar to the system described by Firtha et al. (2008).

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 22 of 36

The system consists of a Xenics NIR camera (Xeva-USB-FPA-1.7-320-TE1-100Hz camera with an InGaAs focal plane array sensor with 2% pixel noise – XEVA 6179; 0.9 μm to 1.7 μm; 320*256 pixel matrix; 12 bit) connected to a Specim N17E spectrograph (Spectral Imaging Ltd., Oulu, Finland) operating in the wavelength range of 900 – 1700 nm with an objective lens (2/3" C-mount broadband coated lenses, Schneider–Kreuznach CINEGON), a 600 mm Y-table gear and stable diffuse 45/0 illumination created by halogen bulbs emitting light in the whole spectral range of the spectrograph. Cooling of the XEVA 6179 camera down to -4 to -10 ° C (269-263° F), which is needed to reduce the noise in the images, is achieved by forced convection (TE-1) cooling. The y table is driven by Isel LF4 mechanics and Isel TMO-4403 (PICMIC) stepping motor that is controlled by textual commands via an RS-485 interface (www.isel.com).

The selection of the appropriate field of view for the samples is achieved by setting the appropriate height of the spectrograph in relation to the sample. Log sizes and therefore the required field of views of 150-450 mm diameter yield a spatial resolution of 0.47 to 1.4 mm. Image focusing is achieved by adjusting the lens. Wavelength focusing needed for a sharp and detailed representation of the wavelengths is performed using the bands created by a Mercury-Cadmium lamp in NIR range by the adjustment of the backfocal length.

The image of the whole setup including the camera with accessories is shown in Figure 8.

Figure 8. Experimental set-up for NIR hyperspectral imaging of wood samples

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 23 of 36

To improve the indirect lightning properties for minimizing inhomogeneous lightning on the wood log surfaces Teflon sheets as reflectors of the halogen light

will be used (Boldrini et al. 2012) (Figure 9).

Figure 9. Ideal indirect illumination for automated inline control by indirect reflected light to minimize surface reflections and spectral artefacts (after

Boldrini et al. 2012).

Both hyperspectral systems, at BOKU and CNR, are of the push-broom type and consist of following components:

• stabilized NIR or visible range light source: illuminates the sample surface • optics: examine/magnify the points of this line • NIR/visible spectrograph: disperses the light from line into its component

wavelengths • NIR/visible photo detector array: digitalizes the rectangle area by given x

spectral and w spatial resolution • Motion system: moving the object (y) with constant velocity and grabbing

I( w, x ) matrixes by given frequency, the Intensity ( w, x, y ) hypercube is acquired

The optimal signal level (white and dark references) is determined by measuring light reflected from a 99 % Zenith Polymer® diffuse reflection standard (SphereOptics, Uhldingen, Germany) and dark signal of the detector without any illuminations (assumed as a background noise).

The spectral calibration is carried out by measurement of Zenith Polymer® diffuse reflection standard containing rare earth metals (SphereOptics, Uhldingen, Germany). Spatial calibration and the validation of the Y-table motion is based on the dedicated samples posessing regular pattern of areas of varying optical properties.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 24 of 36

During each measurement a 3 cm*50 cm Zenith Polymer® diffuse reflection standard rod (SphereOptics, Uhldingen, Germany) will be measured along with the wood log as a spectral reference.

4.5.1.2 Software for NIR hyperspectral system The ARGUS software (Firtha 2010) will control the instrument operation (including camera, motor, cooling) at BOKU, ensure stability and reliability of measurements, eliminate noise and enhance the signal level. The software will be able to reduce data by real-time extraction of tested features. The ARGUS software (Firtha 2010) automatically performs the background removal, treatment of dead pixels and assembly of the hypercubes.

Data mining will be performed by using a custom hyperspectral data processing software CuBrowser developed by Firtha (2012) in Matlab®. It provides several tools suitable for exploration of hyperspectral images and for identification of significant wavelengths attributed to wood defects in different regions of interest. The software package allows pre-processing of spectra, including noise reduction, handling 3D surface topography and roughness, processing of artifacts produced by lightning conditions, Standard Normal Variate normalization, Savitzky-Golay smoothing, first and second derivation, subtraction or linear combination of different wavelengths to enhance the expression of different structures and deficiencies, among others.

The commercially avaliable software packages be also used in alternative to the CuBrowser. The list of such tools will include; PLS_toolbox, MIA_toolbox, Model_exporter, Unscrambler, OPUS and SPSS, among others.

4.5.2 VIS-NIR hyperspectral imaging system at CNR

4.5.2.1 VIS-NIR hyperspectral camera The hyperspectral system avaliable at CNR-IVALSA is a custom build prototype as shown in Figure 10. The optical system consists of spectrograph (Specim V10) , high sensitivity CCD camera (Hamamatsu ORCA-5) and telecentric lenses (Computar TEC-55) . The light source is a halogen bulb emitting light that covers the whole spectral band of the spectrograph. The movement of the hyperspectral system over the measured sample is performed by means of the laboratory CNC machine . Sample is placed on the machine table, when the optical system mechanically connected with the light source is moved over the measured surface. The computer is used for controlling the motion, triggering the camera and collecting raw data. The data are in a form of the 12-bit gray images and are stored on the hard disk for post-processing on the external server.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 25 of 36

Figure 10. Experimental set-up for VIS-NIR hyperspectral imaging of wood samples

In analogy to the NIR system, the VIS-NIR hyperspectral camera captures series of images in the push-broom mode, where each image corresponds to a single line on the measured surface. The pixels intensity in the spectral direction corresponds to the reflectance spectra of the light irradiating the surface. It is possible to determine the spectral map of the evaluated surface by repeating the image acquisition along the motion of the sample. It that case, the map will correspond to the color pattern over the sample surface.

The spatial resolution in direction perpendicular to the movement vector depends on the magnification of the optics used and number of photosensitive elements on the CCD array. The resolution in direction parallel to the movement vector is related to the scanner motion system, movement velocity and frame rate of the CCD Camera.

4.5.2.2 Software for VIS-NIR hyperspectral system The software tools used for VIS-NIR hyperspectral data acquisition and processing will be custom developed by CNR. The National Instruments LabView 14 is foreesen as a development platform. The white reference will be routinely measured before any analysis in order to calibrate the system and will be considered as a 100 % reflectance. The dark image will be acquired after covering the lenses with a wrap. Ten corresponding images will be acquired and averaged

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 26 of 36

for each measurement line in order to increase the signal-to-noise ratio (minimize the measurement error).

The software post-processing images will work in the following cycle:

• open white and dark reference images

• compute relative reflectance image according to equation 1:

DWDIR

−−

= (1)

where; R- reflectance image, I-raw image of sample, D-dark reference image, W-white reference image

• build the hyperspectral cube on the base of all images acquired

• extract spectra corresponding to each spatial pixel

• extract spectral band of interest from the whole spectra

• interpolate results to the vector of 10 nm distances between points

• save results in the hard disk

• alternatively; compute CIE XYZ and/or CIE Lab color coordinates

• alternatively; perform chemometric analysis

4.5.3 Sampling of trees for laboratory trials An extensive set of Norway spruce (Picea abies) samples suitable for tuning the hyperspectral system will be collected from demonstration sites in Austria and Italy. Each sample will be in a form of a disk. These will contain high quality wood samples as well as that possessing various material defects. Wood samples will be gathered in selected distances along the trunk length. Disks after cutting-out from the log will be wrapped with aluminum or vinyl foil to avoid any loss of moisture during storage and transportation, as well as to avoid any wood degradation before characterization. All measurements will be performed immediately after samples delivery to the laboratory. Alternatively, wooden samples will be stored in a deep frozen conditions (-20°C).

4.5.4 Surface conditions and detection performance The extended set of wood samples will be measured with hyperspectral systems including various sample configuration and presentation:

• rough surface with original moisture content at 5 different temperatures (-5, 0, 5, 15, 25° C), including artificial wetting to simluate ice, snow and rain

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 27 of 36

• rough surface at conditioned moisture same temperature (norm climate 20 °C and 60 % air moisture which relates to about 12 % wood moisture)

• grinded surface at same conditioned moisture to assess the effect of surface roughness on the results in relation to the targeted deficits

• different angles of lightning, especially in rough surface conditions; indirect lightning enhanced by teflon reflectors is supposed to remove most of the surface effects of lightning

• different contaminations like soil and/or oil.

This approach will allow the identification and potential mathematical correction(s) for disturbing factors.

4.5.5 Combination of hyperspectral imaging and NIR spectroscopy

4.5.5.1 NIR measurements and data transferability Series of near infrared spectroscopic studies will be carried out at the selected regions of interest in addition to VIS-NIR and NIR hyperspectral imaging. NIR spectroscopy is a proved technology for the analysis of wood and determination of its quality (Tsuchikawa 2007). Both, BOKU and CNR have an extensive experiences in assessing wood quality by NIR technologies (Gindl et al. 2001, Fackler et al. 2006, Fackler et al. 2007, Sandak et al. 2009, Sandak et al. 2011).

The available collections of the spectral databases can be utylized for the quality system callibration and validation. Furthermore BOKU and CNR are also involved in the Trees4future project (P7 under grant agreement n° 284181), where several synergies with regard to the transferability of NIR measurements are existing.

First results achieved within the SLOPE project comparing HSI data and NIR measurements showed, that NIR spectra could be well matched with data gathered by hyperspectral imaging (Zitek et al. 2014) and consequently may improve the reliability of the hyperspectral evaluation.

4.5.5.2 Spectral library of wood characteristics A library of data describing the wood defects will be established by using both NIR and HI, and will be cross-evaluated between the techniques and the different surface conditions of the wood.

4.5.6 Wood defects detection and HI quality index Different multivariate data analyses techniques described in paragraph 3.2.5.2 will be used to identify the spectral signatures of wood deficits. An overall HI quality index for detection and classification will be developed. The index is a numerical value between 0 and 1, when the first indicates a very poor quality log, while “1”

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 28 of 36

indicated the superior resources. Dedicated software will be developed by CNR in order to detect presence of abnormal wood, identify the defect(s) type, and quantify its size and position. The software will be integrated with other modules used for quality sorting according to the SLOPE approach. Several sub-indices will be computed by considering various quality aspects (such as presence of rot, diameter of the heartwood, extent of the reaction wood, etc.). The expert system software module will be used for combining all above indexes and to determine the quality grade or/and optimal destination of the log.

5 Sensors for the prototype

There are several sensor possibly used for the hyperspectral imaging of wood under field conditions. However, the peculiarity related to the extremely harsh working condition when considering tree harvesting in the mountain areas eliminates some of these from the list. The table bellow summarizes hyperspectral sensors identified for the extensive testing campaign.

• Array of photodiodes, 128 elements (http://it.rs-online.com/web/p/array-di-fotorilevatori/7857692/)

• array of Silicon NPN Phototransistor Arrays (http://it.rs-online.com/web/p/array-di-fotorilevatori/6655274/)

• TSL1401CL LineScan Camera (https://www.tindie.com/products/AP_tech/tsl1401cl-linescan-camera-/)

• C12666MA Hamamatsu Micro-spectrometer (http://www.hamamatsu.com/jp/en/C12666MA.html)

• C10988MA-01 Hamamatsu Micro-spectrometer (http://www.hamamatsu.com/jp/en/product/category/5001/4016/C10988MA-01/index.html)

• C11708MA Hamamatsu Micro-spectrometer (http://www.hamamatsu.com/jp/en/product/category/5001/4016/C11708MA/index.html)

• IMSPECTOR V10, Specim spectrograph (http://www.specim.fi/index.php/products/industrial/imaging-spectrographs/vis-vnir)

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 29 of 36

6 . APPENDIX

6.1 Application of Hyperspectral Imaging in different fields

6.1.1 Remote sensing Hyperspectral imaging has its origin and is widely applied in remote sensing applications. In airborne spectral imaging either multispectral sensors with relatively low number of spectral bands are used, or hyperspectral sensors providing more dozens or hundreds of narrow, adjacent spectral bands. It has been applied to airborne mineralogical and chemical mapping (Jones 2010) using airborne HyMap scanner that measures 126 channels covering wavelengths from visible to short wavelength infrared (SWIR) combined with ground measurements of soils and rocks with a ASD portable spectrometer, which covers wavelengths from 350-2500 nm (visible to short wavelength infrared) at 1 nm increments. With an airborne hyperspectral imaging combined with ground-based mapping of spectral signatures also potentially allowed the classification canopy chemical and taxonomic diversity in tropical forests (Asner & Martin 2008, Asner et al. 2011). Selected structures in the urban environments can be identified by selected spectral bands (Heiden et al. 2007), or based on known spectra specific elements in an landscape can be detected (Manolakis et al. 2003) The basis for all mappings is the combination of airborne sensing with ground based mapping and spectral libraries for feature identification (e.g. ASTER spectral library of the NASA). Different types of spectrometers characteristics are existing (Smith 2012).

6.1.2 Environment With airborne hyperspectral imaging relevant vegetation and water resources can be studied (Govender et al. 2007), flood detection and monitoring, detection of water quality, wetland mapping, measures of plant physiology and structure, land-use and vegetation classification, evapotranspiration (Govender et al. 2007).

6.1.3 Soil Hyperspectral imaging has been used to study the spatial distribution of chemical and physical soil characteristics and for the discrimination and classification of horizons, inclusions, particular organic matter (POM), but also elements like Mn, Fe and C in the lab (Buddenbaum & Steffens 2012) and to detect organic structures like wheat root and straw in soil (Eylenbosch et al. 2014).

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 30 of 36

6.1.4 Product and process control Hyperspectral imaging has also a major field of application in the quality control along the material production chain (Gosselin et al. 2011, Duchesne et al. 2012, Kessler 2012).

6.1.5 Application in food quality and safety control Probably the most abundant applications of hyperspectral imaging can be found in food science, e.g. for the control and monitoring of food and feed products (Kim et al. 2002, Mehl et al. 2004, Fernández Pierna et al. 2006, Gowen et al. 2007, Naganathan et al. 2008, Firtha 2009, Sun 2010, Lorente et al. 2012, Felfoldi et al. 2013, Vitale et al. 2013, Wu & Sun 2013b, Wu & Sun 2013a, Huang et al. 2014, Lee et al. 2014). Nicolaï et al. (2007) provide a comprehensive review on NIR and hyperspectral imaging to assess fruit and vegetable quality.

6.1.6 Application in archaeology Vincke et al. (2014) analysed the conservation status of collagen in archeological bone material.

6.1.7 Application in agriculture In a review of Dale et al. (2013), various applications of near-infrared hyperspectral imaging (NIR-HSI) in agriculture and in the quality control of agro-food products are presented.

6.1.8 Application in plant science Application of HI to map the spatially resolved enrichment of N in a plant (Yu et al. 2014) or mapping of quality of medical herbs (Sandasi et al. 2014) were carried out.

6.1.9 In pharmaceutics Review of pharmaceutical applications of vibrational chemical imaging and chemometrics including HIS was compiled by Gendrin et al. (2008)

6.1.10 Medical applications NIR imaging methods do not require ionizing radiation and have great potential for detecting caries lesions (tooth decay) or demineralization on high-risk proximal and occlusal tooth surfaces and at the earliest stages of development (Chung et al. 2011). Also for the histopathological examination of tumor cells it has been successfully applied (Vasefi et al. 2011). A comprehensive review of hyperspectral imaging in medicine is provided by (Lu & Fei 2014)

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 31 of 36

6.1.11 Forensic applications Recent advances in HSI technology for forensic science applications, e.g. the development of portable and fast image acquisition systems, are described by (Edelman et al. 2012)

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 32 of 36

7 References

Agresti, G., Bonifazi, G., Calienno, L., Capobianco, G., Lo Monaco, A., Pelosi, C., Picchio, R. & Serranti, S. (2013) Surface Investigation of Photo-Degraded Wood by Colour Monitoring, Infrared Spectroscopy, and Hyperspectral Imaging. Journal of Spectroscopy, 2013.

Amigo, J.M., Martí, I. & Gowen, A. (2013) Chapter 9 - Hyperspectral Imaging and Chemometrics: A Perfect Combination for the Analysis of Food Structure, Composition and Quality. In: M. Federico (ed.) Data Handling in Science and Technology. Elsevier.

Asner, G.P. & Martin, R.E. (2008) Airborne spectranomics: mapping canopy chemical and taxonomic diversity in tropical forests. Frontiers in Ecology and the Environment, 7, 269-276.

Asner, G.P., Martin, R.E., Knapp, D.E., Tupayachi, R., Anderson, C., Carranza, L., Martinez, P., Houcheime, M., Sinca, F. & Weiss, P. (2011) Spectroscopy of canopy chemicals in humid tropical forests. Remote Sensing of Environment, 115, 3587-3598.

Beleites, C. (2014) hyperSpec Introduction, http://r.adu.org.za/web/packages/hyperSpec/vignettes/introduction.pdf, accessed at 12.01.2015. No.

Bharati, M.H., MacGregor, J.F. & Tropper, W. (2003) Softwood Lumber Grading through On-line Multivariate Image Analysis Techniques. Industrial & Engineering Chemistry Research, 42, 5345-5353.

Boldrini, B., Kessler, W., Rebner, K. & Kessler, R.W. (2012) Hyperspectral imaging: a review of best practice, performance and pitfalls for inline and online applications. Journal of Near Infrared Spectroscopy, 20, 438-508.

Buddenbaum, H. & Steffens, M. (2012) Mapping the distribution of chemical properties in soil profiles using laboratory imaging spectroscopy, SVM and PLS regression. EARSeL EProceedings, 11, 25-32.

Burger, J. & Kaušakytė, A. (2013) Visual Chemometrics – Interactive Software for Hyperspectral Image Exploration and Analysis. 27 March, 2013 Gembloux, BE: SIA BurgerMetrics, Riga, Latvia.

Burger, J.E. & Gowen, A.A. (2011) The interplay of chemometrics and hyperspectral chemical imaging. Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), 2011 3rd Workshop on.

Candolfi, A., De Maesschalck, R., Jouan-Rimbaud, D., Hailey, P.A. & Massart, D.L. (1999) The influence of data pre-processing in the pattern recognition of excipients near-infrared spectra. Journal of Pharmaceutical and Biomedical Analysis, 21, 115-132.

Chung, S., Fried, D., Staninec, M. & Darling, C.L. (2011) Multispectral near-IR reflectance and transillumination imaging of teeth. Biomedical Optics Express, 2, 2804-2814.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 33 of 36

Dale, L., Thewis, A., Rotar, I., Fernandez-Pierna, J.A., Boudry, C., Vidican, R. & Baeten, V. (2012) Chemometric tools for NIRS and NIR hyperspectral imaging. Bulletin of USAVM Cluj Napoca, Agriculture, 69.

Dale, L.M., Thewis, A., Boudry, C., Rotar, I., Dardenne, P., Baeten, V. & Pierna, J.A.F. (2013) Hyperspectral imaging applications in agriculture and agro-food product quality and safety control: A review. Applied Spectroscopy Reviews, 48, 142-159.

Duchesne, C., Liu, J. & MacGregor, J. (2012) Multivariate image analysis in the process industries: A review. Chemometrics and Intelligent Laboratory Systems, 117, 116-128.

Duncker, P. & Spiecker, H. (2009) Detection and classification of Norway spruce compression wood in reflected light by means of hyperspectral image analysis. IAWA journal, 30, 59-70.

Edelman, G.J., Gaston, E., van Leeuwen, T.G., Cullen, P.J. & Aalders, M.C.G. (2012) Hyperspectral imaging for non-contact analysis of forensic traces. Forensic Science International, 223, 28-39.

ElMasry, G. & Sun, D.-W. (2010) Principles of hyperspectral imaging technology. Hyperspectral imaging for food quality analysis and control, 3-43.

Eylenbosch, D., Fernández Pierna, J.A., Baeten, V. & Bodson, B. (2014) Detection of wheat root and straw in soil by use of Near Infrared hyperspectral imaging system and Partial Least Square discriminant analysis. 13th Congress of the European Society for Agronomy.

Fackler, K., Gradinger, C., Hinterstoisser, B., Messner, K. & Schwanninger, M. (2006) Lignin degradation by white rot fungi on spruce wood shavings during short-time solid-state fermentations monitored by near infrared spectroscopy. Enzyme and Microbial Technology, 39, 1476-1483.

Fackler, K., Schwanninger, M., Gradinger, C., Srebotnik, E., Hinterstoisser, B. & Messner, K. (2007) Fungal decay of spruce and beech wood assessed by near-infrared spectroscopy in combination with uni-and multivariate data analysis. Holzforschung, 61, 680-687.

Felfoldi, J., Baranyai, L., Firtha, F., Friedrich, L. & Balla, C. (2013) Image processing based method for characterization of the fat/meat ratio and fat distribution of pork and beef samples. Progress in Agricultural Engineering Sciences, 9, 27-53.

Fernandes, A., Lousada, J., Morais, J., Xavier, J., Pereira, J. & Melo-Pinto, P. (2013a) Comparison between neural networks and partial least squares for intra-growth ring wood density measurement with hyperspectral imaging. Computers and Electronics in Agriculture, 94, 71-81.

Fernandes, A., Lousada, J., Morais, J., Xavier, J., Pereira, J. & Melo-Pinto, P. (2013b) Measurement of intra-ring wood density by means of imaging VIS/NIR spectroscopy (hyperspectral imaging).

Fernández Pierna, J.A., Baeten, V. & Dardenne, P. (2006) Screening of compound feeds using NIR hyperspectral data. Chemometrics and Intelligent Laboratory Systems, 84, 114-118.

Fernández Pierna, J.A., Vermeulen, P., Amand, O., Tossens, A., Dardenne, P. & Baeten, V. (2012) NIR hyperspectral imaging spectroscopy and chemometrics for the detection of undesirable substances in food and feed. Chemometrics and Intelligent Laboratory Systems, 117, 233-239.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 34 of 36

Firtha, F. (2009) Detecting moisture loss of carrot samples during storage by hyperspectral imaging system. Acta Alimentaria, 38, 55-66.

Firtha, F. (2010) Argus hyperspectral acquisition software, ftp://fizika2.kee.hu/ffirtha/Argus-CuBrowser.pdf, accessed 20.11.2014. No.

Firtha, F. (2012) CuBrowser hyperspectral data processing algorithm ftp://fizika2.kee.hu/ffirtha/Argus-CuBrowser.pdf, accessed 20.11.2014. No.

Firtha, F., Fekete, A., Kaszab, T., Gillay, B., Nogula-Nagy, M., Kovács, Z. & Kantor, D.B. (2008) Methods for improving image quality and reducing data load of NIR hyperspectral images. Sensors, 8, 3287-3298.

Gat, N. (2000) Imaging spectroscopy using tunable filters: a review. AeroSense 2000. International Society for Optics and Photonics.

Geladi, P., Grahn, H. & Burger, J. (2007) Multivariate images, hyperspectral imaging: background and equipment. Techniques and Applications of Hyperspectral Image Analysis, 1-15.

Geladi, P., Sethson, B., Nyström, J., Lillhonga, T., Lestander, T. & Burger, J. (2004) Chemometrics in spectroscopy: Part 2. Examples. Spectrochimica Acta Part B: Atomic Spectroscopy, 59, 1347-1357.

Gendrin, C., Roggo, Y. & Collet, C. (2008) Pharmaceutical applications of vibrational chemical imaging and chemometrics: A review. Journal of Pharmaceutical and Biomedical Analysis, 48, 533-553.

Gindl, W., Teischinger, A., Schwanninger, M. & Hinterstoisser, B. (2001) The relationship between near infrared spectra of radial wood surfaces and wood mechanical properties. Journal of Near Infrared Spectroscopy, 9, 255-261.

Gosselin, R., Rodrigue, D. & Duchesne, C. (2011) A hyperspectral imaging sensor for on-line quality control of extruded polymer composite products. Computers & Chemical Engineering, 35, 296-306.

Govender, M., Chetty, K. & Bulcock, H. (2007) A review of hyperspectral remote sensing and its application in vegetation and water resource studies. Water Sa, 33.

Gowen, A., O'Donnell, C., Cullen, P., Downey, G. & Frias, J. (2007) Hyperspectral imaging–an emerging process analytical tool for food quality and safety control. Trends in Food Science & Technology, 18, 590-598.

Grahn, H. & Geladi, P. (2007) Techniques and applications of hyperspectral image analysis: John Wiley & Sons.

Heiden, U., Segl, K., Roessner, S. & Kaufmann, H. (2007) Determination of robust spectral features for identification of urban surface materials in hyperspectral remote sensing data. Remote Sensing of Environment, 111, 537-552.

Huang, H., Liu, L. & Ngadi, M.O. (2014) Recent Developments in Hyperspectral Imaging for Assessment of Food Quality and Safety. Sensors, 14, 7248-7276.

Jones, M., Cudahy, T., Thomas, M., Hewson, R. (2010) Airborne and ground-based spectral surveys map surface minerals and chemistries near Duchess, Queensland. 19th World Congress of Soil Science, Soil Solutions for a Changing World 1 - 6 August 2010. Brisbane, Australia. : Published on DV.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 35 of 36

Kessler, R.W. (2012) Prozessanalytik: Strategien und Fallbeispiele aus der industriellen Praxis: John Wiley & Sons.

Kim, M., Lefcourt, A., Chao, K., Chen, Y., Kim, I. & Chan, D. (2002) Multispectral detection of fecal contamination on apples based on hyperspectral imagery: Part I. Application of visible and near-infrared reflectance imaging. TRANSACTIONS-AMERICAN SOCIETY OF AGRICULTURAL ENGINEERS, 45, 2027-2038.

Lee, H., Kim, M.S., Jeong, D., Delwiche, S.R., Chao, K. & Cho, B.-K. (2014) Detection of Cracks on Tomatoes Using a Hyperspectral Near-Infrared Reflectance Imaging System. Sensors (Basel, Switzerland), 14, 18837-18850.

Lied, T.T. & Esbensen, K.H. (2001) Principles of MIR, multivariate image regression: I: Regression typology and representative application studies. Chemometrics and Intelligent Laboratory Systems, 58, 213-226.

Lied, T.T., Geladi, P. & Esbensen, K.H. (2000) Multivariate image regression (MIR): implementation of image PLSR—first forays. Journal of Chemometrics, 14, 585-598.

Lorente, D., Aleixos, N., Gómez-Sanchis, J., Cubero, S., García-Navarrete, O.L. & Blasco, J. (2012) Recent advances and applications of hyperspectral imaging for fruit and vegetable quality assessment. Food and Bioprocess Technology, 5, 1121-1142.

Lu, G. & Fei, B. (2014) Medical hyperspectral imaging: a review. Journal of Biomedical Optics, 19, 010901-010901.

Manolakis, D., Marden, D. & Shaw, G.A. (2003) Hyperspectral image processing for automatic target detection applications. Lincoln Laboratory Journal, 14, 79-116.

Mehl, P.M., Chen, Y.-R., Kim, M.S. & Chan, D.E. (2004) Development of hyperspectral imaging technique for the detection of apple surface defects and contaminations. Journal of Food Engineering, 61, 67-81.

Naganathan, G.K., Grimes, L.M., Subbiah, J., Calkins, C.R., Samal, A. & Meyer, G.E. (2008) Visible/near-infrared hyperspectral imaging for beef tenderness prediction. Computers and Electronics in Agriculture, 64, 225-233.

Nicolaï, B.M., Beullens, K., Bobelyn, E., Peirs, A., Saeys, W., Theron, K.I. & Lammertyn, J. (2007) Nondestructive measurement of fruit and vegetable quality by means of NIR spectroscopy: A review. Postharvest Biology and Technology, 46, 99-118.

Prats-Montalbán, J.M., de Juan, A. & Ferrer, A. (2011) Multivariate image analysis: A review with applications. Chemometrics and Intelligent Laboratory Systems, 107, 1-23.

Sandak, A., Sandak, J. & Negri, M. (2011) Relationship between near-infrared (NIR) spectra and the geographical provenance of timber. Wood science and technology, 45, 35-48.

Sandak, A., Sandak, J., Pradzynski, W., Zborowska, M. & Negri, M. (2009) Near infrared spectroscopy as a tool for characterization of wood surface. Folia Forestalia Pol, 40, 31-40.

Sandasi, M., Vermaak, I., Chen, W. & Viljoen, A.M. (2014) Hyperspectral Imaging and Chemometric Modeling of Echinacea—A Novel Approach in the Quality Control of Herbal Medicines. Molecules, 19, 13104-13121.

Smith, R.B. (2012) Introduction to hyperspectral imaging. No.

SLOPE - Integrated proceSsing and controL systems fOr sustainable forest Production in mountain arEas – FP7-NMP-2013-SME-7 --604129 WP 4 – Multi-sensor model-based quality of mountain forest production Deliverable 4.04 Establishing hyperspectral measurement protocol

Co-founded by the European Commission

Page 36 of 36

Sun, D.-W. (2010) Hyperspectral imaging for food quality analysis and control: Elsevier.

Thumm, A., Riddell, M., Nanayakkara, B., Harrington, J. & Meder, R. (2010) Near infrared hyperspectral imaging applied to mapping chemical composition in wood samples. Journal of Near Infrared Spectroscopy, 18, 507.

Tsuchikawa, S. (2007) A Review of Recent Near Infrared Research for Wood and Paper. Applied Spectroscopy Reviews, 42, 43-71.

Vasefi, F., Najiminaini, M., Ng, E., Chamson-Reig, A., Kaminska, B., Brackstone, M. & Carson, J. (2011) Transillumination hyperspectral imaging for histopathological examination of excised tissue. Journal of Biomedical Optics, 16, 086014-086014-11.

Vidal, M. & Amigo, J.M. (2012) Pre-processing of hyperspectral images. Essential steps before image analysis. Chemometrics and Intelligent Laboratory Systems, 117, 138-148.

Vincke, D., Miller, R., Stassart, É., Otte, M., Dardenne, P., Collins, M., Wilkinson, K., Stewart, J., Baeten, V. & Fernández Pierna, J.A. (2014) Analysis of collagen preservation in bones recovered in archaeological contexts using NIR Hyperspectral Imaging. Talanta, 125, 181-188.

Vitale, R., Bevilacqua, M., Bucci, R., Magrì, A.D., Magrì, A.L. & Marini, F. (2013) A rapid and non-invasive method for authenticating the origin of pistachio samples by NIR spectroscopy and chemometrics. Chemometrics and Intelligent Laboratory Systems, 121, 90-99.

Wise, B.M., Gallagher, N.B., Bro, R., Shaver, J.M., Windig, W. & Koch, R.S. (2006) PLS_Toolbox Version 4.0 for use with MATLAB™. No.

Wu, D. & Sun, D.-W. (2013a) Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review — Part II: Applications. Innovative Food Science & Emerging Technologies, 19, 15-28.

Wu, D. & Sun, D.-W. (2013b) Hyperspectral Imaging Technology: A Nondestructive Tool for Food Quality and Safety Evaluation and Inspection. In: S. Yanniotis, P. Taoukis, N.G. Stoforos & V.T. Karathanos (eds.) Advances in Food Process Engineering Research and Applications. Springer US.

Yu, K.-Q., Zhao, Y.-R., Li, X.-L., Shao, Y.-N., Liu, F. & He, Y. (2014) Hyperspectral Imaging for Mapping of Total Nitrogen Spatial Distribution in Pepper Plant. PLoS ONE, 9, e116205.

Zhang, X., Liu, F., He, Y. & Li, X. (2012) Application of hyperspectral imaging and chemometric calibrations for variety discrimination of maize seeds. Sensors, 12, 17234-17246.

Zitek, A., Firtha, F., Böhm, K., Parrag, V., Sandak, J. & Hinterstoisser, B. (2014) Inspection of log quality by hyperspectral imaging. Fifth IASIM conference in spectral Imaging, IASIM-14, DEC 3-5, 2014. Rome: IASIM, Book of Abstracts, on USB stick.