accurate binary representation of singulated geranium-cutting images

Computers and Electronics in Agriculture, 6 ( 1992 ) 319-332 319 Elsevier Science Publishers B.V., Amsterdam

Accurate binary representation of singulated geranium-cutting images

L. Wallace and W. Simonton Georgia Experiment Station, Department of Agricultural Engineering, University of Georgia,

1109 Experiment Street, Griffin, GA 30223-1797, USA

(Received 17 May 1991 )

ABSTRACT

Wallace, L. and Simonton, W., 1992. Accurate binary representation of singulated geranium-cutting images. Comput. Electron. Agric., 6:319-332.

Various methods of image processing were implemented to accurately distinguish geranium cuttings from a black conveyor surface background. Seven methods were compared: global thresholding, basic local thresholding, a more complex method of local thresholding termed the quadtree method, Otsu's threshold selection, KSW threshold selection, histogram flattening and high-pass filtering. The resulting binary images were judged subjectively based on their similarity to the gray-scale image and preservation of detail. Objective performance of the seven methods was based on a both a shape measure and a region-uniformity measure. The three methods which preserved the most detail upon conversion from gray-scale data to binary without significant introductions of noise and loss of dimensional accuracy were the high-pass filter, the basic local threshold and the quadtree method.

INTRODUCTION

Machine vision has been shown to be an effective tool for assisting proper robot operation in the typically nonuniform systems found in agriculture (Slaughter and Harrell, 1987; Davis, 1991 ). In the Mechatronics Laboratory of the Georgia Station, a robotic workcell has been developed for automated processing of singulated geranium cuttings for propagation. The workcell has been evaluated and performed comparably to laborers in a commercial green- house (Simonton, 1990). For the robot to successfully process a cutting it is necessary that, by means of machine vision, various parts of the cutting be identified. Important cutting parts are the main stem, petioles and leaves (Fig. l ). With this information, the robot can be automatically controlled to cor- rectly process each individual plant.

The most common way to extract an object from the background is global thresholding. Global-threshold selection involves choosing a gray level t such that all gray levels greater than t are converted into the object (plant), while

0168-1699/92/$05.00 © 1992 Elsevier Science Publishers B.V. All fights reserved.

320 L. WALLACE AND W. SIMONTON

leaf (~ver/aePPin~g ~ ~ ~ ~

leaves~ ~ // /~ j'

- , Vll I region

main stern "~

Fig. 1. Typical morphology of a zonal geranium with associated plant parts marked.

all other gray levels are converted into background, assuming a dark conveyor surface as background. A single threshold is chosen for the whole image. This method of creating a binary image can cause a loss of detail which may lead to errors in object or part classification (Simonton and Pease, 1990 ). To min- imize the errors in identification of plant parts, particularly the main stem growing tip, a more accurate binary image was desired. For any imaging system which relies on binary data the conversion from gray-scale information to binary should be performed in a manner to retain detail and preserve dimensional accuracy. The primary objective of this study was to evaluate alternatives to global thresholding for creating a binary image of vegetative cuttings places randomly on a conveyor surface. In Materials and Methods the various alternative methods employed will each be discussed and it includes a brief description of the performance measures used to evaluate the results of the methods implemented in this study. Results and Discussion will compare results of each of the methods on 50 test samples. The Conclusion will include an evaluation of the methods.

MATERIALS AND METHODS

The computer (Heurikon HK68/V3e) used in this study was a VMEbus, 25-MHz MC68030/68882, 4-MB system running a real-time multitasking operating system (Microware 0S9/68k). Software was developed using C. The vision hardware was comprised of two DataCube Maxvid¢o VMEbus boards serving input/output (D ig imax)and frame-storage (Framestore) functions. A monochrome RS-170 CCD camera (Jensen WV-CD22 ) was the

BINARY REPRESENTATION OF SINGULATED GERANIUM-CUTTING IMAGES 321

Fig. 2. Gray-level image of a 'Crimson Fire' geranium cutting. Note the optics have been confi- gured for maximum separation between plant and background.

Fig. 3. Binary image of the 'Crimson Fire' cutting as created by global threshold at 190.


vision sensor employed. Pixel resolution was 512 × 484 with 256 gray levels. The lens was a 25-mm unit with the aperture set at 3.5 for a high contrast image. The camera was mounted 110 mm above a conveyor surface. The conveyor was a commercial unit with a black fairly non-reflective belt 350 mm in width. This arrangement yielded a two-dimensional system resolution of 0.72 mm horizontally and 0.56 mm vertically. Light level was approximately 1400 lx as provided by overhead fluorescent and incandescent fixtures.

Figure 2 reflects a typical gray-level image of a geranium cutting, the object used in this study. Note the high contrast between plant and background. Seven methods of creating a binary image from an RS-170 gray-level image were implemented. Each method was then evaluated on criteria presented by Sa- hoo et al. (1988). The criterion functions used to evaluate performance were a shape measure and a region-uniformity measure. These measures were performed on 50 cuttings which were stored as gray-level images. The images were comprised of three different geranium varieties: 'Crimson Fire', 'Sincer- ity' and 'Valerie'. The methods employed were then compared to one another and particularly to the image created by a global threshold (Fig. 3 ), the method used in the robotic workcell. Each of the methods will be explained below.

Local thresholding

A local threshold is performed by dividing the image into several regions. A threshold level is determined by examining the gray-level histogram of the image included in each region. The gray-level histogram is smoothed by an averaging process so that maximum and minimum values would more accurately represent the peaks and valleys in the histogram (Sahoo et al., 1988). A minimum value between the two peaks can be chosen as the threshold value for each region individually (Fig. 4). In this study, the threshold value t was limited to a certain range ( 180-240 of 256 gray levels) to avoid sharp differ- ences in the plant image, such as in petiole width.

U c- O

° °" dll, , u. II l h , , i i , , , , ,

A 255 Gray level

Fig. 4. Histogram of region in local thresholding procedure. The arrow reflects the threshold selection of the next region to be converted to a binary image.

BINARY REPRESENTATION OF SINGULATED GERANIUM-CUTTING IMAGES

Quadtree method

323

The quadtree method is a local thresholding technique which does not use constant size regions. With the quadtree method, the image is subdivided into quadrants if its gray level standard deviation is high (over 10% of the mean value ), repeating this process for each quadrant. This yields a decomposition of the image into blocks (quadrants, quadrants of quadrants .... ), each having a low standard deviation (under 10%). The image approximation obtained by recursive subdivision into quadrants is referred to as a Q_image (Wu et al., 1982). Blocks with a low standard deviation are classified as homogeneous. I fa block is homogeneous then a threshold can be selected for that block. The threshold-selection technique implemented in this study involved examining the histogram for that block. The minimum value closest to the peak created by background pixels was selected as the threshold (Fig. 5 ). If there were no background pixels located in the block then the highest level within the threshold range (180-240) was selected. The length of the smallest edge of any block was constrained to 15 pixels.

Otsu's threshold selection

Another method of selecting optimal thresholds to extract objects from the background was presented by Otsu (1979). The Otsu technique is dependent solely on the gray-level histogram for a priori knowledge. The method selects a threshold t so that the interclass variance between dark and bright regions is maximized. The method is based upon the assumption that well-thresh- olded classes would be separated in gray levels, and conversely, a threshold giving the best separation of classes in gray levels would be the best threshold. The Ostu method may be used as an alternative to locating the bottom of the valley in a gray-level histogram since the histogram is not always bimodal and the valley is not easily distinguished.

Ill I~. I I . . . . h , . . .

255

G r a y l e v e l

Fig. 5. Histogram of smallest block in the quadtree method. The arrow reflects the threshold selection of the next block to be converted to a binary image.

324

KSW threshold selection

L. WALLACE AND W. SIMONTON

Kapur et al. ( 1985 ) suggested a method of threshold selection based on the entropy of the histogram concept originally proposed by Pun ( 1981 ). This method defines two probability distributions, one defined for discrete values I to a threshold t and the other for values from t+ 1 to 255. In this study, the possible values for t were limited to a range from 180 to 240. The method then defines an entropy related to the range in intensities for each probability distribution, above and below the threshold t. The t which maximizes the distinction of the two entropies is selected as the optimal threshold. The KSW method differs from Pun's in that it does not have unnecessary bias in the threshold selection. This method, like Otsu's, was developed as an alternative to histogram valley-seeking techniques.

Histogram flattening

The principle of histogram flattening is to reduce the frequency of numerous intensity values and increase the frequency of relatively rare values. This can be done by dividing the cumulative frequency distribution - - the sum of the values up to each intensity level - - into a large number of increments for relatively frequent values and a smaller number of increments for rare values. This technique has the effect of sharpening images, because relatively rare intensity values, such as those at edges, are emphasized at the expense of common intensity values, such as those inside regions (Cohen and Feigenbaum, 1989).

The algorithm began the flattening procedure at an intensity level which represented the uppermost intensity of the majority of background pixels. Figure 6 displays the gray-level histogram of the 'Crimson Fire' cutting in Fig. 2. A typical intensity for beginning the flattening procedure was 140. This procedure created an accentuated plant image with only four intensities. From this image an algorithm based upon several heuristics transformed the four- intensity image into a binary image.

oc , 11111 l lh , , , , . . . . . I

A 2ss G r a y l e v e l

Fig. 6. Histogram of the gray-level 'Crimson Fire' image in Fig. 2.


High-pass filtering

Approaches other than thresholding can be used to separate objects from the background in a gray-level image. One such approach is that of edge detection using a high-pass filter. A high-pass filter emphasizes areas of high frequency (i.e., areas with abrupt changes in intensity such as edges). The high-pass filter implemented in this study depended on a 3 × 3 neighborhood using a convolution of the following array:

{ - 1 , - 1 , - 1 , - 1,9,- l , - 1,-- 1 , - l}

The convolution algorithm simply adjusts each pixel according to the weighted average of the gray level values of the neighboring points (Charniak and McDermott, 1985 ). The filter results in an outline of the gray-scale image.

A binary image of the plant cutting was required and not merely the edge outline. Therefore, the convolution algorithm was modified to create a binary image rather than a trace of the edges. The binary conversion relied upon a condition to convert each pixel to plant if the filtering result exceeded a set value (180-240), or otherwise assign it to background. A second filter was performed on the image which eliminated a majority of the background noise and also smoothed out some of the plant detail. This 'smooth binary' filter simply relied on the average of a 3 X 3 neighborhood to determine if a pixel was isolated or surrounded by like pixels (Cohen and Feigenbaum, 1989 ).

Measures of evaluation

A performance measure was required to assist in objectively judging the methods' effectiveness. Sahoo et al. ( 1988 ) implemented two measures which compare the binary and gray-level images to rate various thresholding techniques. These measures for evaluating shape and region uniformity were uti- lized to provide an objective rating of each method.

The shape measure was calculated by assigning a gradient value to each pixel based upon its 3 × 3 neighborhood. This measure was multiplied by - 1 if the pixel has a gray value lower than the average of its neighbors ( 3 X 3 ) or when the pixel had been converted to background, the pixel intensity was less than the threshold value. The shape measure S was computed using the formula

S= ~, Sgn(f(x,y)--fN(x,y)) A(x,y) Sgn( f ( x , y ) - t ) (1) (x, .v)

with (x, y) as pixel coordinates, where f (x , y) is the gray level of the pixel (x, Y),fN~x.y) is the average gray value in the neighborhood N(x, y), t is the threshold value of the image, and

1 if z > 0 Sgn(z )= _1 if z < 0 (2)


The gradient value A(x, y) of the pixel (x, y) is computed by the following formula

4 11/2 3(x, Y) = I ~ D 2 + x/2Dl (D3 + D4)-x /2D2(D3 +D4) (3)

Lk= 1

where O1 = f ( x + 1, y) - f ( x - 1, y) 02 = f ( x , y - 1 ) - f ( x , y + 1 ) D3=f(x+ 1,y+ 1 ) - - f ( x - 1,y-- 1) D 4 = f ( x + 1, y - 1 ) - f ( x - 1, y+ 1 )

The uniformity measure is based on the variance within each region. This measure is dependent upon the gray-level image and the size of the region. Applied to this study, each binary image consisted of two regions, plant and background. The uniformity measure was originally designed for a measure of accuracy for image segmentation by Levine and Nazif (1985). The uniformity measure U was calculated by the following formula

U= a2a + a2p (4)

where tr2~ = Y, [ f ( x, y ) - lti ] 2 (x, y)~Ri

E f¢x, y) (x, y)eRi

l g i - A i '

Ri = Region (either background or plant ) At = Number of pixels in R, i = B, P (Background, Plant).

RESULTS AND DISCUSSION

Table 1 shows typical results on a cutting comparing the alternative methods with global thresholding at three levels. The values of the performance measures are relative to each image and simply allow the methods to be ranked. The following describes results from each alternative.

Local threshold

The basic local thresholding technique yielded accurate and reliable performance both on the basis of the shape and uniformity measures and subjec- tive comparisons between the gray-scale and resulting binary images. Local thresholding needed plant features in the binary image that the global threshold frequently converted to background (Fig. 7). The time required for this method to convert a gray-scale image to binary through software was an av-

BINARY REPRESENTATION OF SINGULATED GERANIUM-CUTTING IMAGES

TABLE 1

Shape and uniformity measures for a typical geranium-cutting image

327

Method Time Shape-measure Rank Uniformity Rank a Combined (s) rank

HIpass 36.56 155 145.64 8 133 618 764 8 8 Local 3.98 105 950.77 6 92 028 523 6 6 Q_image 29.72 117 836.70 7 86 751 799 5 6 Global 200 1.14 65 083.36 5 84 969 260 4 4 Flatten 20.28 - 8 5 598.08 2 95 354 178 7 4 Otsu 2.44 35 891.95 4 84 554 478 3 3 Global 190 1.14 - 18 090.39 3 84 152 995 0 2 KSW 3.12 - 9 7 414.16 1 84 179 303 1 0 Global 180 1.12 - 120 679.27 0 84 335 775 2 0

aRank 0-8, 8=best .

erage of approximately 4.5 s and the method ranked third among all proce- dures according to the performance measures (Table 2 ). The computational time made local thresholding desirable to consider for the real-time robot workcell environment, although the method would still need to be implemented in hardware. As to difficulties, on occasions when a petiole was the only plant part in a local window, a high threshold would be selected which

Fig. 7. Binary image of the 'Cr imson Fire ' cut t ing by means of local thresholding.


TABLE 2

Average time and rank according to the shape and uniformity measures of 50 cuttings (Varieties: Crimson Fire, Sincerity, Valerie)

Method Avg. rank Avg. time (0-8) (s)

Hlpass 7.70 38.6 Q_image 5.61 31.2 Local 5.54 4.4 Global 200 4.27 1.4 Flatten 3.61 23.2 Otsu 2.88 2.7 Global 190 2.58 1.4 KSW 2.56 3.4 Global 180 1.94 1.4

created a thin binary petiole thus losing dimensional accuracy. The threshold selection algorithm is crucial for the success of this method.

Quadtree method

The quadtree method's performance was dependent upon the smallest size of block that was allowed. With the smallest block edge restricted to fifteen pixels, the time required for thresholding was approximately 32 s. This min- imal size of fifteen pixels was optimal for avoiding jagged plant edges and gaining the accuracy desired. Because the blocks become increasingly smaller around the edges of the plant, the detail around the edges was preserved (Fig. 8 ). Of all methods, the quadtree method produced the most accurate results based on visual comparison with the gray-level image (Fig. 9). Overall, the quadtree method ranked second of the alternatives tested based on the shape and uniformity measures. The primary limitation of this method, with the exception of computation time, was that this technique left some of the plant outline with a less defined, fuzzier edge than did the other methods.

Otsu's threshold selection

The Otsu method yielded poor results when implemented both as a threshold selection for the local boxes and when used as a means of selecting a global threshold. Within range limitations (180-240), the Otsu method tended to choose either 180 or a threshold value that was too high and resulted in a very thin plant. At times the resulting image would include petioles that were dis- connected either from the leaves or the thinner portions of the petiole. This method ranked sixth among the all tests.


Fig. 8. Example showing use of the quadtree method as partially completed.

Fig. 9. Binary image of the 'Crimson Fire' cutting by the quadtree method.

330 L. WALLACE AND w. SIMONTON

Fig. 10. Edge outline created by a high-pass filter.

Fig. 11. Binary image of the 'Crimson Fire' cutting by the high-pass filter and smoothing algorithm.

BINARY REPRESENTATION OF SINGULATED GERANIUM-CUTTING IMAGES 3 31

KSW threshold selection

The KSW method tended to select a threshold value that was consistently too low, resulting in a thicker image. The thicker image produces increased occlusion and reduces the spatial accuracy of forks in the petioles. The KSW method, like Otsu's method, was implemented for selecting global and local threshold values. The KSW method was compared with the other methods when implemented for the global threshold selection technique. This technique ranked among the lowest due to the thicker images it produced.

Histogram flattening

The resulting image of the histogram-flattening algorithm consisted of a background of intensity 0, a 'halo' around the plant of intensity l, a plant outline of intensity 6 and the plant itself of intensity 4. The algorithm did not accurately choose the uppermost background intensity level on some plants, resulting in a broader outline of the plant. If the camera aperture was reduced, creating more obvious shading in the plant image, this method was more dependable. This method required an average of 23 s computation time. Ac- cording to the shape and uniformity measures this method ranked fifth among the methods implemented in this study. While producing a binary image that adequately represented many cuttings,this method was not consistent in that it varied in ranking from 3 to 7 on the test images. This variation of its performance among the tested images indicated poor reliability.

High-pass filter

The high-pass filter (Fig. 10) followed by the 'smooth binary' filter (Fig. l 1 ) yielded exceptional results according to the shape and uniformity measures, ranking highest of all the methods on both the shape measure and the uniformity measure. When compared visually to the other methods this method produced sporadic holes in the leaf regions caused by extreme shading, or naturally darkened bands in the leaves, in the gray-level image. The 'smooth binary' algorithm unfortunately smeared detail in the stem regions while it eliminated background noise. Time required for each filter pass was approximately 19 s, making the total time average to 38 s. High rankings by the shape and uniformity measures could be attributed to the fact that the evaluations were also based upon the 3 X 3 neighborhood of each pixel.

C O N C L U S I O N

Image processing is a key element in the success of machine vision as it prepares the image for analysis and classification. Effective classification of

3 3 2 L. WALLACE AND W. SIMONTON

plant parts is critical in the robotic workcell for geranium processing, and the binary image generated from the gray-level image must represent the plant accurately with the detail preserved. Numerous methods were explored which produced a binary image, either by thresholding or a form of edge detection. The alternatives to global thresholding were evaluated by two performance measures and rated accordingly. The high-pass filter generated the most accurate image based on the performance measures. The quadtree method and local thresholding method performed on a nearly equal level. These three alternatives all ranked above the present method of a global threshold.

If any of the evaluated methods could be implemented into hardware, the high-pass filtering method would be the most dependable. It provides the needed detail retention and a true-to-size binary image. The quadtree method may be more difficult to implement using hardware. This method would be successful depending on the method of threshold selection for each block. The Q_image provides the needed detail as well as a true-to-size image of the cutting. The local threshold method would yield dependable results if the camera aperture was maintained at an optimal setting and used a reliable threshold selection technique, and this method could be implemented in hardware somewhat simpler than the quadtree method.

REFERENCES

Charniak, E. and McDermott, D., 1985. Introduction to Artificial Intelligence. Addison-Wes- ley, Reading, MA, pp. 99-113, 162.

Cohen, P. and Feigenbaum, E., 1989. The Handbook of Artificial Intelligence, Vol. III. Addi- son-Wesley, Reading, MA, pp. 206-224.

Davis, P.F., 1991. Orientation-independent recognition of chrysanthemum nodes by an artificial neural network. Comput. Electron. Agric., 5:305-314.

Kapur, J., Sahoo, P. and Wong, A., 1985. A new method for gray-level thresholding using entropy of histogram. Comput. Vision Graphics Image Process., 29: 273-285.

Levine, M.D. and Nezif, A.M., 1985. Dynamic measurement of computer generated image seg- mentations. IEEE Trans. Pattern Anal. Mach. Intell., 7:155-164.

Otsu, N., 1979. A threshold selection method for gray-level histograms. IEEE Trans. Systems Man Cybern., 9( 1 ): 62-66.

Pun, T., 1981. Entropic thresholding: A new approach. Comput. Vision Graphics Image Pro- cess., 16: 210-231.

Sahoo, Soltani and Wong, 1988. SURVEY: A Survey of Thresholding Techniques. Comput. Vision Graphics Image Process., 41: 233-260.

Simonton, W., 1990. Automatic Geranium Stock Processing in a Robotic Workcell. Trans. ASAE, 33(6): 2074-2080.

Simonton, W. and Pease, J., 1990. Automatic Plant Feature Identification on Geranium Cut- tings Using Machine Vision. Trans. ASAE, 33 (6): 2067-2073.

Slaughter, D.C. and Harrell, R.C., 1987. Color vision in robotic fruit harvesting. Trans. ASAE, 30(4): 1144-1148.

Wu, Hong and Rosenfeld, 1982. Threshold selection using quadtrees. IEEE Trans. Pattern Anal. Mach. Intell., 4: 90-94.

accurate binary representation of singulated geranium-cutting images

Documents