Classification Framework for Nailfold Capillary Microscopy Images

Chia-Hsien Wen Wei-Duen Liao

Dept. of Computer Science and Information Management Providence University

Shalu, Taichung 43301 Taiwan E-mail: chwen@pu.edu.tw

Kuan-Ching Li Dept. of Computer Science and Information Engineering

Providence University Shalu, Taichung 43301 Taiwan

E-mail: kuancli@pu.edu.tw

Abstract - Nailfold capillary microscopy examination has been used since late 1950s as a non-invasive in-vivo technique for diagnosing and monitoring connective tissue disease in adults. Disorders such as Raynaud’s phenomenon, progressive systemic sclerosis, and rheumatoid arthritis were detected in more than 80% of adult patients, by analyzing such high resolution images. In this research paper, we propose a framework to classify nailfold capillary microscopy images into SLE (Systemic Lupus Erythematosus) and PSS (Progress Systemic sclerosis) diseases. Based on statistical data collected in Taichung Veteran’s General Hospital (TCVGH), Taiwan, higher percentage of adult patients are find with SLE patients, when compared with PSS.

According to ARA (American Rheumatism Association), patients’ outside features are obviously. The PSS patients are the same way to finding the features. And in the same time, we refer from doctor’s idea add other conditions to help judging. In the first step, RP is the most important element. Most of the SLE and PSS features are raised by RP. In the other way, the features can be found in both of SLE and PSS patients. In order to divide the SLE and PSS patients into the correct class, other conditions are existed in necessary.

I. INTRODUCTION

Analysis of nailfold capillaries microscopy images have been used since late 1950s, and accepted as a non-invasive in-vivo technique to diagnose adults with connective tissue diseases. By using this technique, disorders such as Primary Raynaud’s phenomenon, progressive systemic sclerosis, dermatomyositis, and rheumatoid arthritis and other related abnormalities can be detected in more than 80% of adult patients, while rheumatic diseases in most children patients. Such technique has proven its efficiency to provide necessary information to aid the diagnosis of a number of disorders both in adults and children.

There are a number of ways to obtain Nailfold capillaries microscopy images, and they are threefold. The first and simplest way is to fix a high resolution digital camera and take an image directly over desired region where the physician intends to diagnose. Not only simple and easy, this method is widely used worldwide. Alternatively, some researchers have proposed to acquire these images by selecting frames from a recorded video. The last approach is to utilize several video frames from a sequence and integrate them into a single image, averaging the temporal variability and build up a “mosaic” of the whole area under investigation [6]. This method may

generate nailfold images of high resolution than previous approaches.

The Division of Allergy, Immunology and Rheumatology, Department of Internal Medicine, Taichung Veterans General Hospital (TCVGH), Taiwan, is the largest authority in Taiwan when speaking of treating immunological diseases in both adults and children. Their patients are numerous across Taiwan, and they have traced and kept this large amount of anamneses for more than two decades, being one the best sources for our research [8].

In this research, we propose a framework to classify nailfold capillaries microscopy images using a number of image processing techniques, and identify whether such high resolution images is a normal or abnormal. If latter situation happens, the proposed investigation may classify and discern the image into SLE (Systemic Lupus Erythematosus) and PSS (Progress Systemic Sclerosis) diseases. Features of these two different types of diseases shown in nailfold images may probably be the same, though they may affect and show in different regions of human body.

The standard of clinical diagnosis SLE was defined by ARA in 1972 [5], and the symptoms are: Malar rash, Discoid lupus, Photosensitivity, Oral ulcer, Arthritis, Serositis, Renal disorders, Neurologic disorders, Hematologic disorder, Anti-dsDNA antibody, anti-SM antibody, Antiphospholipid antibody and finally, Antinuclear antibody.

PSS is often found in Frigid Zone. Particularly in Taiwan, patients diagnosed with this disease are of high death rate, since there is neither medication nor medical procedure to completely cure. The most notable characteristic among patients owing this disease is that patient’s finger skin become harder.

The remainder of this paper is as follows. In section 2, we present backgrounds of this research, while in section 3 the framework for nailfold image classification and method for image preprocessing. We present some experimental results in section 4, and finally, some concluding remarks and future work in section 5.

II. BACKGROUND

Nailfold image was used from 1950s, it’s helpful to early detect diseases, and trace them. In the paper, it was mainly used to find some features and count them. Raynaud’s phenomenon is the main result caused by SLE and PSS. Even though, it’s impossible to treat completely.

In addition to morphological abnormalities, measurements have been made of capillary density, capillary blood flow velocity, and the diffusion of dyes through the capillary wall. The normal capillary landscape is a uniform palisade of loops which are homogeneous in size and morphology. Several researchers have observed that this pattern was completely disorganized in the nailfold in the presence of certain diseases.

M. Cutolo et al.[1] defined five taxonomies from nailfold images. As following, 1) Enlarged/giant vessel 2) hemorrhages point 3) Loss of capillaries 4) disorganization vessel 5) Ramified/bushy capillaries. The main idea is from features’ morphology. The five taxonomies can help us to select features from new patient’s nailfold images, and try to count them. When the features counted done, from numbers, it’s helpful to recognize the patient’s disease situation. The situation was classed in three levels, early, active, and late patterns. The features have different statistical data, from the result, it understands to class different pattern.

In pre-process steps are trying to transfer color images into gray level images, because gray image is easier to process than color one. Before thinning vessel’s path, the image was threshold and then transferred into binary image. The Skeleton algorithm was used to detect vessel’s path.

According to R.F. Chang et al. [2] proposed five distinctions to record vessel’s information for analysis. They are vessel to volume ratio, numbers of vessel, number of bifurcation, mean of radius, tortuosity measure. They are using on 3-D breast lesion ultrasound image.

The main idea to process images, we take from [3][4], in order to acquire vessel’s path skeleton. The distance from border to median is used to compute the diameter of vessel, that can show that if the capillary is enlarge or giant. In the same way, others capillaries phenomenon, maybe show the skeleton of capillaries in different result image.

III. PROPOSED FRAMEWORK

Figure 1 shows the framework for classifying nailfold images. Nailfold capillary microscopy images are acquired during patient’s diagnosing session with his physician in TCVGH, and are transmitted via Internet to high performance servers located in Providence University, where image processing application is being designed and developed. As the computation has finalized, some statistical analysis are performed to generate a classification report regarding to the image processed.

Figure 1. Image classification framework.

A. Image Acquisition

Nailfold capillary images are captured by a set of CCD equipments shown in Figure 2. Before taking images, mineral oil is dropped and rubbed on fingers to reduce light reflection. Eight image frames, from patient’s fingers except pollexes, are captured for each patient. Figure 3(a) shows four image frames taken from the right hand of a patient. Each selected CCD image frame is transferred to an AVI screen capturer to get a 640*480 image with useful features. Figure 3(b) shows the AVI image captured from the left-upper image frame of Figure 3(a). Then this color image frames are transformed into monochrone images for classification.

Figure 2. Nailfold image acquisition equipment including microscopy, printer, finger stand, and CCD camera.

(a) (b)

Figure 3. (a) Sample CCD nailfold images captured from the right hand of a patient. (b) is the AVI image captured from

left-upper one of (a).

B. Image Processing Methodology

Each vessel loop in a nailfold image is processed as an “object” or a “feature”, since vessel loops contain important information for classification. The features caused by RP are in five different types. According to M. Cutolo et al.[1], these features are named as follows:

1. Enlarged/giant capillaries, 2. Hemorrhages points, 3. Loss of capillaries, 4. Disorganization capillaries, 5. Ramified/bushy capillaries.

Figure 4 shows some nailfold image with number features.

Internet

TCVGH Server PU Server & Nodes

Acquiring Image

GRID

Allocate

Framework

Integrate

Results Remote Devices

Figure 4. Numbering features of vessel loops according to their feature types.

The first job of classification is to find the morbid

vessels and organizations. Pre-processing on a nailfold image turns its features nitid. In this paper, each monochrome inailfold image is equalized and transformed into a binary image with thresholding technique. Then the Skeleton algorithm is utilized to mark vessel’s basis.

Figure 5. A nailfold image with thinned vessels after being processed by Skeleton algorithm.

Basically, the Skeleton algorithm is an iterative

process and each iteration is composed of two steps [3, 5] described as follows. For each step, each pixel of the binary nailfold image is processed with a 3x3 mask shown in Figure 6.

Figure 6. A 3x3 mask used by the Skeleton algorithm.

Step 1. For each pixel, the follwing conditions are checked:

(a) 2<= N(p1) <=6 (b) S(p1) = 1;

(c) p2*p4*p6 = 0; (d) p4*p6*p8 = 0;

where N(p1) is the number of nonzero neighbors of p1, that is, N(p1) = p2+p3+…..+p8+p9. S(p1) is the count of zero-to-one changes of pn to pn+1, where 1 < n < 9.

If all conditions (a)-(d) are satisfied, the pixel is

flagged and the next pixel is checked. After all pixels of the binary image are processed, the flagged pixel will be deleted and then start to execute step 2. Step 2. The procedure of step 2 is the same as step 1 except the following conditions to be checked:

(a) 2<= N(p1) <=6 (b) S(p1) = 1; (c’) p2*p4*p8 = 0; (d’) p2*p6*p8 = 0;

The iteration is stop when there is no more pixel be

flagged and the thinned vessel’s basis is obtained. Figure 5 shows a nailfold image with its thinned vessels.

After finding the thinned vessel basis, resulting data will be inserted into NN (Neural Network) to calculate the values of features and send the patient’s images into the right classification.

IV. EXPERIMENTAL RESULTS

The steps of experimental process in the proposed framework as shown in Figure 7 are as follows:

1. Patient’s image data is captured; 2. Transform the original color image into

monochrome image; 3. Pre-process the monochrome image; 4. Put result data image into Neural Network; 5. Get the final result; 6. Match the result with gold standard; 7. Record the result in database.

Figure 7. Steps for image processing.

Figure 8 shows several approaches used to process images in step 2 and step 3. In fact, not all of the methods are invoked. Alternatives are adopted case by case.

Based on sample nailfold images from 74 patients, almost 50% are gold standard images. Experimental results reveals that the Skeleton algorithm does not adapt to all nailfold images. It is most suitable for nailfold capillary image with features 1-3 in figure 4. In cases of image with

p9 p2 p3

p8 p1 p4

p7 p6 p5

Internet

PC in TCVGH

Device in PUCapture Image

Storage

Upload & Transfer

Select or Manage

feature 4 or feature 5, they are too complex to be recognized by the proposed process in this paper. Most of the current researches in this field can not absolutely solve the recognition problem, identifying nailfold images of all diagnostic patients.

(a) (b) (c)

(d) (e) (f)

Figure 8. Different image processing approaches over same image. (a) original image; (b) monochrome image; (c) brightness-reduced image; (d) equalized image; (e) binary image; (f) color the white region with red.

Table 1 shows the feature patterns as RP in progress which is resulted in our experiments. In early stages of diseases cause RP, enlarged/giant capillary was often found. More haemorrhages spots and loss of capillaries appeared later. A few disorganization and ramified/bushy capillaries were advent. In late stages, most of enlarged/giant capillaries became disorganized with ramified/bushy capillaries.

Table 1. Feature patterns as RP in progress.

Early pattern

Active pattern

Late pattern

1.Enlarged/giant ● ● ▲

2.Haemorrhages ○ ● ▲

3.Loss of capillaries ▲ ○ ○ 4.Disorganization ▲ ○ ●

5.Ramified/bushy ※ ▲ ●

●: More, ○: Moderate, ▲: Less, ※: None

Processed nailfold images and their results will be archived in databases after classification. In order to be retrieved efficiently, the patient’s ID number, birthday, and appointment date are used as a composite primary key of the record.

V. CONCLUSIONS AND FUTURE WORK

A classification framedwork for nailfold capillary microscopy images is presented in this paper. It is used for detecting features caused by RP. Simple images featured with single vessel loops may be recognized easier. Unfortunately, complex images with disorganization capillaries or bushy capillaries are not solved yet. We still devoted ourselves in refining the image process algorithm at present.

As future work, an efficient computing system is necessary, in terms of clinical diagnosis. We expect to build a trustable system to provide doctors a helpful diagnostic support and makes effective plan for treatment, in order to detect disease and prescript ways for prevention.

ACKNOWLEDGMENTS

This research is sponsored in part by National Science Council (NSC), Taiwan, under grants NSC95-2218-E-007-025, NSC95-2221-E-126-006-MY3 and NSC96-2221-E-126-004-MY3. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSC.

REFERENCES [1] M. Cutolo, C. Pizzorni, M. Tuccio, A. Burroni, C. Craviotoo,

M. Basso, B. Seriolo and A. Sulli, “Nailfold vidocapillaroscopic patterns and serum autoantibodies in systemic sclerosis”, Rheumatology, Vol. 43, pp. 719-726, 2004.

[2] R.F. Chang, S.F. Huang, K.M. Woo, Y.H. Lee, and D.R. Chen, “Computer Algorithm for Analyzing Brest Tumor Angiogenesis using 3-D Power Doppler Ultrasound”, Ultrasound in Med. & Biol, Vol. 32, No. 10, pp 1499-1508, 2006.

[3] R.C. Gonzalez and R.E. Woods, “Digital Image Processing”, Addison Wesley, pp 491-495, 1993.

[4] T.Y. Zhang and C.Y. Suen, “A fast parallel algorithm for thinning digital pattern,” Communications of the ACM, Vol. 27, No. 3, 1984, pp. 236-239.

[5] C.M. Passas, R.I. Wond, and M. Peterson, “A comparison of the specificity of the 1971 and 1982 American Rheumatism Association criteria for the classification of systemic lupus erythematosus”, Arthritis Rheum, Vol. 28, pp 620-623, 1985.

[6] P.D. Allen, C.J. Taylor, A.L. Herrick, T. Moore, “Image Analysis of Nailfold Capillary Patterns From Video Sequences”, Medical Image Computing and Computer-Assisted Intervention 1999.

[7] P.D. Allen, C.J. Taylor, A.L. Herrick, T. Moore, “Image Analysis of Nailfold Capillary Patterns”, Medical Image Understanding and Analysis 1998.

[8] K.-C. Li, C.-N. Chen, T.-Y. Hsieh, C.-H. Wen, J.-L. Lan, D.-Y. Chen, and C.-Y. Tang, “Towards Design of a Nailfold Capillary Microscopy Image Analysis and Diagnosis Framework using Grid Technology”, in Journal of High Speed Networks, vol. 16, no. 1, pp. 81-89, IOS Press, 2007.