[ieee 2014 third international conference on agro-geoinformatics - beijing, china...

Download [IEEE 2014 Third International Conference on Agro-Geoinformatics - Beijing, China (2014.8.11-2014.8.14)] 2014 The Third International Conference on Agro-Geoinformatics - A new approach for measuring 3D digitalized rape leaf parameters based on images

Post on 11-Mar-2017




1 download

Embed Size (px)


  • A New Approach for Measuring 3D Digitalized Rape Leaf Parameters based on Images

    FANG Yihang, LIN Chengda*, ZHAI Ruifang, TANG Yao, WANG Xingyu College of Resource and Environment

    Huazhong Agricultural University Wuhan, China


    AbstractAt present, measurement method of crop leaf area mainly proceeds in two-dimensional way, which may lead to certain degree of damage. Therefore, it will be of great significant to propose a non-contact method for the measurement of leaf area. In this paper, pictures of rape are obtained from different viewangles with mobile camera, and then three-dimensional digitization of rape is realized by SFM(Structure From Motion) and MVS(Multiple View Stereo) technologies. We conduct a case study and establish a three-dimensional modeling of rape leaf by applying the NURBS surface fitting. Finally, surface area of rape leaf is automatically calculated, and then compared to the area which is obtained by traditional digital image measurement methodology, in this way it realizes non-destructive detection of rapte growth modeling for precision agriculture. According to the experimental results, the the method of 3D digitalized rape leaf measurement based on images is supported.

    Keywords3D leaf model; structure from motion (SFM); multiple view stereo(MVS); leaf area measurement; precision agriculture

    I. INTRODUCTION The leaf is a significant organ for transpiration,

    photosynthesis and synthetic organics, and the developmental situation and size of leaves show a huge impact on the growth and development, stress resistance and yield, and meanwhile, it can also guide the cultivation density and fertilization level, plant physiology, etc. Furthermore, rape is one of the most significant crops in China [1]. Meanwhile, it is the significant raw materials of fodder, chemical engineering and energy; it also plays huge strategic and economic roles in the national economy. Consequently, it is quite urgent and necessary to propose an approach for measuring the rape leaf directly under the natural growing conditions. The development of digital agriculture and precision agriculture is of great significance, which may provide real-time and accurate basic information about the key growth stage of plants for digital agriculture and precision agriculture, so as to provide support decisions for agricultural production.

    Traditionally, the leaf area was mainly measured in two-dimensional way, for instance planimeter method, transparent squares method, weighing method, empirical formula method, leaf platometer, digital image processing method, etc. However, these methods are all destructive detection, which

    may cause certain damages to the leaves, destroy the continuous growth, and further impact the continuity of crop experiment. It not only wastes time and energy, but also impacts the accuracy and precision with different conditions and subjective attitudes. At present, the new measurement method mainly gains the three-dimensional point cloud data of plants in non-contact method, and then three-dimensional model is established from the three-dimensional point cloud data to measure the parameters of morphological characteristics for the plants. For instance, the achieving of three-dimensional point cloud data with laser scanning, but it has inherent defects, for instance, being short of leaf color information and texture information, expensive laser scanning device, as well as the sensitivity to experimental environment [2,3]. However, the machine vision-based form detection and modeling technology is an alternative approach of achieving the three-dimensional model of specific plants [4,5]. Researchers have already proposed several measures of plant reconstruction, including the surface section-based reconstruction, outline-based reconstruction, three-dimensional point cloud reconstruction based on stereo vision [6,7]. Furthermore, the three-dimensional point cloud reconstruction based on stereo vision can solve the problems occurring in the previous methods, and it can achieve three-dimensional model of plants, as well as the color and texture information of leaves cheaply, conveniently and non-destructively.

    Fig.1 shows the workflow of three-dimensional modeling of rape leaf based on stereo vision and non-destructive measurement technology. At first, rape photos shall be obtained from different view angles, and then SFM technology [8]is applied for generating the sparse rape three-dimensional point cloud data and camera parameters from a series of non-calibration photos. On that basis, regional growing is conducted with the data generated by MVS [9] and SFM technology for producing dense three-dimensional point cloud. Later, the dense three-dimensional point cloud is obtained with artificial segmentation. Finally, NURBS surface fitting is implemented for the dense three-dimensional point cloud, and then the area of surface model is measured. Furthermore, the experimental result is compared to that obtained from traditional digital image measurement methodologies.

    *Corresponding author: LIN Chengda, linchengda@mail.hzau.edu.cn

  • Images

    Keypoints detection(SIFT Keypoint Detection)

    Keypoints match(SIFT Keypoint Detection)


    Leaf surface fitting(NURBS)



    Produce 3DDense piont cloud

    Produce 3DSparse piont cloud

    Fig. 1. The workflow chart


    A. Detection and Matching of Key Point Features SIFT[10] (Scale-invariant feature transform) is an

    algorithm for detecting the local features. It remains the same in rotation, scaling, and brightness variation, with certain stability in change of view, affinity variation and noise. Therefore, SIFT algorithm is adopted to detect and match the key point features. It mainly consists of four steps: 1. Detection of scale space extreme value. In the scale space, the potential interest point with invariable scale and rotation is detected with Gaussian differential function. 2. Location of key point. On the location of interest point, the location and scale of key points shall be determined. 3. Confirmation of direction. Each key point shall be assigned with direction based on the local gradient direction of image. 4. Description of key point. Local gradient of image is measured within the field of each key point.

    B. Three-dimensional Digitization of Rape with SFM and MVS Technology The process of calculating the three-dimensional

    information of scenes with image sequence or video is an inverse process of the imaging process. Generally, there are two steps of the calculation process: SFM (Structure From Motion) and MVS (Multi-view Stereo) . SFM, which also known as calibration, refers to a sparse three-dimensional point cloud calculated by image sequence and camera parameters of each picture (internal parameters, orientation and location). After the camera parameters of each image are obtained, the dense three-dimensional point cloud can be calculated by MVS according to the dense matching of pixel points between images.

    Fig. 2. SFM Schematic Diagram

    SFM mainly refers to the process of seeking for matching points from image sequences of different angles and then calculating the corresponding three-dimensional point location and camera parameters of image according to the matching points. The theoretical foundation of SFM (Fig.2) is the geometric principle of perspective projection, namely establishing the relation between two-dimension and three-dimension by introducing a perspective projection. Suppose there is a point X in the three-dimensional space, and its corresponding two-dimensional pixel on image I is x, then, there will be a perspective projection matrix P= K[R/t] (K is the internal parameter of picture, R is the orientation of camera, and t is the camera position), which may get the projection equation x=PX established. The principle of SFM is to extract the matching two-dimensional feature point from pictures taken from different angles, namely x1, x2, ... (matching points in different images). The three-dimensional information of two-dimensional matching point, as well as the internal parameters, orientation and position of each image is calculated with the corresponding two-dimensional matching point and projection equation. In Fig.4 (a), the recovery of sparse three-dimensional point cloud data and camera parameter of each image (Bundler output result) is described. The camera parameter and sparse three-dimensional point cloud obtained from each image through the calculation of sparse matching point of two-dimensional image. Since the three-dimensional point cloud is too sparse, only simple object models can be generated (for instance, regular building, furniture, etc.). Therefore, dense three-dimensional point shall be reconstructed for complicated objects (such as plants), and dense three-dimensional point cloud requires the MVS technology.

    MVS mainly generates corresponding point cloud or grid model with the calibrated image sequence as the input. MVS mainly consists of the following types: (1) method based on voxel, which divides the space of rebuilding object into small cubes (voxel), and the matching degree of images may determine the staying of each voxel; (2). Strain grid method, the visual hull of objects can be obtained through the outlines of objects in image as the initial mesh model, and then the mesh can be transformed to the reach surface of object by taking advantage of the matching degree of images; (3). With the depth map, the corresponding depth map of each picture is calculated according to the pixel point matching among different images, and then, it is combined to be unified three-dimensional point cloud data; (4). The rebuilding object su