Multiquadric Spline-Based Interactive Segmentation ofVascular Networks

Sachin Meena1, V. B. Surya Prasath1, Yasmin M. Kassim1, Richard J. Maude6,7,8, Olga V. Glinskii2,3

Vladislav V. Glinsky2,4, Virginia H. Huxley3,5, Kannappan Palaniappan1

Abstract— Commonly used drawing tools for interactiveimage segmentation and labeling include active contours orboundaries, scribbles, rectangles and other shapes. Thin ves-sel shapes in images of vascular networks are difficult tosegment using automatic or interactive methods. This paperintroduces the novel use of a sparse set of user-defined seedpoints (supervised labels) for precisely, quickly and robustlysegmenting complex biomedical images. A multiquadric spline-based binary classifier is proposed as a unique approach forinteractive segmentation using as features color values and thelocation of seed points. Epifluorescence imagery of the duramater microvasculature are difficult to segment for quantitativeapplications due to challenging tissue preparation, imagingconditions, and thin, faint structures. Experimental resultsbased on twenty epifluorescence images is used to illustratethe benefits of using a set of seed points to obtain fast andaccurate interactive segmentation compared to four interactiveand automatic segmentation approaches.

I. INTRODUCTION

Over the past two decades interactive methods for clinicaland biomedical image segmentation have been investigatedsince the pioneering work of Live-Wire, Live-Lane [1] andIntelligent Scissors [2]. Fully automatic image segmentationis essential for quantitative analysis but remains an unsolvedproblem, so user driven interactive methods continue to be apowerful alternative when extremely precise segmentation isrequired. However, manual methods although routinely usedare tedious, time-consuming, expensive, inconsistent betweenexperts and error prone. In semi-supervised interactive seg-mentation the goal is for the user to provide a small amountof partial information or hints for an automatic algorithmto use in order to produce accurate boundaries suitable forthe user. The coupled interaction between the user providedinput and the semi-supervised segmentation algorithm shouldbe minimal and robust.

This research was supported in part by the Award #1I01BX000609 fromthe Biomedical Laboratory Research & Development Service of the VAOffice of Research and Development (VVG), the National Cancer Instituteof the National Institutes of Health Award #R01CA160461 (VVG) and#R33EB00573 (KP). Mahidol-Oxford Tropical Medicine Research Unit isfunded by the Wellcome Trust of Great Britain.

1Computational Imaging and VisAnalysis Lab, Department of ComputerScience, 2Research Service, Harry S. Truman Memorial Veterans Hospi-tal, Columbia, MO 65201 USA, 3Department of Medical Pharmacologyand Physiology, 4Department of Pathology and Anatomical Sciences and5National Center for Gender Physiology, University of Missouri-Columbia,MO 65211 USA. 6Centre for Tropical Medicine and Global Health, NuffieldDepartment of Medicine, University of Oxford, Oxford, UK, 7Mahidol-Oxford Tropical Medicine Research Unit, Faculty of Tropical Medicine,Mahidol University, Bangkok, Thailand, 8Harvard TH Chan School ofPublic Health, Harvard University, Boston, USA.

(a) Input with seeds (b) Groundtruth (c) MQ Segmentation

Fig. 1. The proposed interactive MQ segmentation result using 29seeds produces accurate results. (a) Dura mater vessel image 012706-ERBKO-05 (contrast enhanced for visualization), (b) ground-truth (GT)vessel segmentations provided by an expert physiologist. (c) result of MQsegmentation without pre- or post-processing (Dice 0.8872).

Commonly used drawing tools for interactive segmenta-tion interfaces include active contour or boundary drawing,scribbles to identify foreground and background regions, andrectangles to outline the object of interest. In the case ofcomplex anatomy, tissue structures and imaging conditions,even such interactive segmentation tools would still requirea significant amount of user intervention to achieve accuratesegmentation results. An example of a difficult case is thevessel network segmentation for the epifluorescence imageof dura mater microvasculature shown in Figure 1. Sincethe vessel structures are thin, manually drawing contours,scribbles or rectangles using current interactive segmentationtools, is tedious, slow and often inaccurate, creating a needfor better semi-automated delineation tools [3], [4]; the latterincludes survey of software tools for reconstructing curvilin-ear structures. Our motivation is to develop quantitative toolsto study the effects of hormone therapy on angiogenesis andvascular remodeling of capillaries [3], [4].

In this paper we propose a new paradigm of using onlylabeled seed points, that do not need to carefully followcenterlines or boundaries for user assisted segmentation ofdifficult biomedical imagery with curvilinear structures. Seedpoints are much easier and more intuitive for the userto provide labeled inputs and requires minimal interactioncompared to contours, lines, rectangles or other shapes.We introduced point-based segmentation using elastic bodysplines and Gaussian elastic body splines in [5]–[7] andshowed their advantages for segmenting natural imageryusing an average of less than ten seed points. In this paper, wefocus on characterizing the utility of using interactive seedpoints for segmenting thin vessels in epifluorescence imageryand extend the basis set to multiquadric splines. Figure 1shows a case of using 9 foreground and 20 background seedpoints to produce a highly accurate segmentation result with

978-1-4577-0220-4/16/$31.00 ©2016 IEEE 5913

a Dice value of 88.7 percent based on scoring the overlapwith the manual ground-truth.

Many techniques exist for automatic segmentation ofvessels, especially in angiogram imaging for characterizingneurological, retinal, and heart regions [8]. The majority ofthese are fully automatic segmentation methods which do notrequire any user intervention. Interactive segmentation meth-ods specialized for thin curvilinear structures provide flexi-bility to the user in defining and refining object boundaries.Interactive segmentation based on techniques such as Graph-Cut [9] and Random Walker [10] are popular approacheswhere the user provide scribbles on the object of interest andthe background. In Live-Wire [1] and Intelligent Scissors [2]the user carefully draws along the object boundary edges ina semi-automated manner. Being edge-based methods theyperform poorly in the presence of weak edges and noise.Recently in [5] and [6] we proposed using sparse set of seedpoints to perform a user driven image segmentation. Sparseinputs leads to a very fast image segmentation process.

In this work we utilize multiquadric (MQ) splines forpoint-based interactive segmentation of thin curvilinearstructures in epifluorescence imagery. Unlike other semi-automated tools [4] our interactive segmentation frameworkrequires only a few labeled seed points and can obtainvessel segmentations under inhomogeneous background, par-tially diffused vessels and uneven contrast. We frame theinteractive image segmentation task as a semi-supervisedinterpolation problem which uses MQ as the basis functionin the governing interpolation equation. This work is part ofa system that builds upon our previous work on multi-focusfusion and edge preserving smoothing [11]–[13].

The rest of the paper is organized as follows. Section IIintroduces our multiquadric splines-based interactive imagesegmentation framework. Section III provides detailed ex-perimental results on epifluorescence imagery of mice duramater capillaries and quantitative performance comparisonwith other interactive and global thresholding methods fol-lowed by conclusions.

II. MULTIQUADRIC SPLINES FOR INTERACTIVE IMAGESEGMENTATION

A. Interactive segmentation with splines

Epifluorescence imagery of tissues have high variabilityin both arteriole and venule vessel boundaries. We con-sider semi-automated segmentation framework based on userdefined seed points to obtain reliable segmentations. Forthis purpose we learn the parameters for a binary classifierfunction df (~x) of the following form from the user suppliedinputs,

df (~x) =m∑i=1

aixi + b+

N∑j=0

g(~x− ~xj)cj . (1)

such that,N∑j=0

cjxj,k = ~0 k = 1, . . . , 5 (2)

where ~x = [x1, x2, . . . , x5]T and xi is the ith feature, m

is the number of features (m = 5 in our case), cj is thespline coefficient and ai and b are coefficients of the linearterm, j is the feature index, N is the total of foregroundand background seed points and g is the multiquadric splinefunction given by,

g(~x− ~xj) =√d2(~x, ~xj) +R2, (3)

where d is the distance between the data points ~x and ~xj . Inour case d2 = ‖~xi − ~xj‖2 and R2 is a user defined positiveconstant. This function is similar to the extensor function forinterpolation [14]. Equations in (1) constitute a linear systemof equations and the constraint in (2) ensures that the systemof equations has a unique solution. In equation (1) the firstterm is the linear part of the interpolation function while thesecond term represents the non-linear part of the interpolationfunction. For the task of interactive segmentation df (~x) islearned from the user labeled seed points such that df (~x)) =+1 for the foreground pixels and df (~x) = −1 for thebackground pixels.

Once the MQ interpolation spline function (1) is learned,we can threshold the interpolation function df (~x) at zero andassign label `(~x),

`(~x) =

{foreground, if df (~x) ≥ 0background, if df (~x) < 0.

(4)

For the interactive image segmentation task we use fiveimage features at each pixel, namely three color features(Red, Green, Blue) converted to the LAB color space and twocoordinate locations of each seed pixel. For the interpolationtask the use of coordinate location as a feature is critical asit effects the influence of seed points and provides geometricspatial regularization.

B. Multiquadric splines

In this work we use multiquadric (MQ) splines as theyprovide robust interpolation of data points in high dimen-sional spaces. Let ~w be the vector of all the MQ coefficientsgiven as,

~w =[~cTF ~cTB ~aT b

](5)

where ~cF and ~cB are coefficients corresponding to the Nf ,foreground and Nb background seed pixels.

~cF =[c1 . . . cNf

]T, ~cB =

[c1 . . . cNb

]T, ~a =

[a1 . . . a5

]TThe MQ coefficients are estimated by solving the matrixequation,

~w = L−1~Y (6)

where,

L =

[K PPT O

], K =

[GFF GFB

GBF GBB

](7)

GFF (~r) =

G11(r11) . . . G1f (r1f )...

...GNf1(r11) . . . GNff (rNff )

(8)

5914

with rij = d(~xi − ~xj) where d is the distance between thefeature vectors ~xi and ~xj , d2 = ‖~xi − ~xj‖2. GFF is thematrix of MQ (3) spline functions defined only over theforeground pixels.

G(i, j) = g(~xi − ~xj) =√d2(~xi, ~xj) +R2

Similarly GFB , GBB and GBF are also defined. We have,

P =

[PF IFPB IB

], (9)

where the identity matrices are given by,

IF =[I1 . . . INf

], IB =

[I1 . . . INb

], (10)

with I is 5× 5 identity matrix, and

PF =

x11 . . . x15...

...xNf1 . . . xNf5

(11)

where xij is the ith foreground pixel for jth feature. ~IF and~IB are vectors of 1. PB is similarly defined. The vector ~Y ,

~Y =[~YF

T ~YBT ~OT

](12)

~YF =[1 . . . 1

]T, ~YB =

[−1 · · · − 1

]T, ~O =

[0 . . . 0

]Tconsists of ~YF with values +1 for the foreground seed pointsand ~YB with values -1 for the background seed points, ~O isa vector of zeros. We refer to [5] and [6] for more detailsfor solving the above matrix problem (6).

III. EXPERIMENTAL RESULTS

The experiments were performed using high resolutionepifluorescence images of mice dura mater acquired using avideo microscopy system (Laborlux 8 microscope from LeitzWetzlar, Germany) equipped with 75 watt xenon lamp andQICAM high performance digital CCD camera (QuantitativeImaging Corporation, Burnaby, Canada) with 0.56 micronper pixel resolution. Our dataset consists of 10 wild-type(WT), and 10 knock-out (KO) mice dura mater epifluo-rescence images stained using Alexa Fluor 488-conjugatedsoybean agglutinin (SBA lectin). We collected manuallydrawn ground-truth (GT) with supervision from an expertphysiologist for benchmarking the segmentation methods.Note that the ground truth for these images reported inthe Table I has since been updated. The overall biologicalmotivation is to quantify the role of estrogen receptors (ERβ)in microvasculature remodeling in WT and KO mice.

Figure 2(a,f,k) shows three example epifluorescence im-ages of mice dura mater (2 WT, 1 KO) vasculature networksused in the experiment along with a sparse set of userprovided seed points sampling the foreground (yellow) andbackground (red). Figure 2(b,g,l) shows the gold standardGT binary image (foreground vessels in white and back-ground in black). We show a comparison of our proposedmultiquadric (MQ) spline-based semi-supervised interpola-tion (Figure 2(c,h,m)) with interactive segmentation methods

TABLE ICOMPARISON OF DIFFERENT SEGMENTATION METHODS ON 20

(10-KNOCK-OUT (012706-ERBKO) + 10-WILD-TYPE

(012606-ERBWT)) MICE DURA MATER EPIFLUORESCENCE IMAGES.WE SHOW THE DICE VALUES FOR OUR PROPOSED MULTIQUADRIC

SPLINE INTERACTIVE SEGMENTATION METHOD, COMPARED WITH

AUTOMATIC THRESHOLDING AS WELL DIFFERENT INTERACTIVE

SEGMENTATION APPROACHES. HIGHER DICE VALUES INDICATE BETTER

RESULTS COMPARED TO MANUAL GOLD-STANDARD GROUND-TRUTH

SEGMENTATIONS.

No. Type Niblack Otsu Random Graph MQ[11] Walker [10] Cut [9]

1 KO-01 0.6324 0.3038 0.2177 0.7301 0.70922 KO-03 0.5456 0.6813 0.3404 0.8528 0.80313 KO-04 0.6196 0.6282 0.3176 0.8023 0.84364 KO-05 0.7970 0.5780 0.5145 0.7097 0.83675 KO-06 0.7452 0.5999 0.6331 0.8080 0.86416 KO-08 0.6875 0.6406 0.5322 0.8154 0.85577 KO-09 0.4363 0.4907 0.2085 0.6018 0.79728 KO-10 0.5736 0.6714 0.5293 0.7259 0.74979 KO-15 0.7909 0.6238 0.5721 0.7660 0.858710 KO-19 0.7882 0.6640 0.6388 0.7128 0.726111 WT-01 0.6665 0.4903 0.6063 0.7582 0.764712 WT-03 0.4095 0.4410 0.3824 0.6766 0.737513 WT-04 0.5726 0.4974 0.3623 0.6818 0.738114 WT-05 0.7589 0.6099 0.5844 0.8126 0.863715 WT-06 0.8467 0.6673 0.6343 0.8520 0.846616 WT-07 0.8296 0.6348 0.7835 0.9448 0.922417 WT-08 0.7436 0.6108 0.7746 0.8441 0.855518 WT-09 0.7116 0.5457 0.4694 0.7010 0.786919 WT-12 0.6774 0.5689 0.8506 0.7540 0.841120 WT-14 0.8772 0.6520 0.6915 0.8094 0.7979Average 0.6855 0.5800 0.5328 0.7680 0.8099

based on Random Walker (RW) [10] (Figure 2(d,i,n)) andGraph-Cut (GC) [9] (Figure 2(e,j,o)). As can be seen, ourmethod obtains good segmentations overall. The RandomWalker interactive segmentation method fails to capturebranching vessels and obtains spurious foreground regions.The Graph-Cut based approach fails to segment diffuse ves-sel boundaries, in particular venules where the vessels havevery low contrast. Our approach obtained good segmentationresults while requiring very few seed points for foregroundvessels. The last column shows an image where all threemethods produce low Dice scores, though our approach stillachieves better segmentation.

To evaluate the performance of different segmentationmethods quantitatively, we use the Dice similarity coefficient,

Dice(P,Q) =2|P ∩Q||P |+ |Q|

,

where P and Q are the representations of automatic andground-truth (GT) segmentations. Values closer to one indi-cate better performance in terms of the physiologist expertverified gold standard. Table I shows the dice values for10 wild-type and 10 knock-out mice dura mater epifluores-cence images. Our method in general outperforms relatedinteractive segmentation methods as well some well-knownautomatic thresholding methods [11] from the literature.Note that we applied no pre- or post-processing methodsto generate these segmentation results.

5915

(a) 012606-ERbWT-07 (9 fg, 18 bg) (b) Ground-truth (c) MQ Dice:0.9423 (d) RW Dice:0.7030 (e) GC Dice:0.9283

(f) 012706-ERbKO-06 (14 fg, 23 bg) (g) Ground-truth (h) MQ Dice:0.8303 (i) RW Dice:0.3404 (j) GC Dice:0.7886

(k) 012606-ERbWT-04 (15 fg, 30 bg) (l) Ground-truth (m) MQ Dice:0.6849 (n) RW Dice:0.1863 (o) GC Dice:0.6412

Fig. 2. Quantitative comparison of semi-automated interactive segmentation results shown for three sample microvasculature images. From left to right:Input images (012606-ERbWT-07, 012706-ERbKO-06, 012606-ERbWT-04) with user selected foreground (fg) and background (bg) seeds, ground truthmask, Multiquadric (proposed method) MQ, Random walk [10] RW, Graph-cut [9] GC. No pre- or post-processing was performed to generate thesesegmentation results. Our proposed MQ method obtains better segmentations without spurious foreground segments and the Dice values indicate we obtaincloser results to GT.

IV. CONCLUSIONS

In this paper, we considered using a seed point-basedinteractive image segmentation method for microvasculaturenetwork extraction from epifluorescence imagery. Using abinary classification function based on multiquadric splinesour method was able to obtain reliable segmentations usingonly sparse seed points from the user. Seed points are mucheasier and more intuitive for the user to provide. In thiscase for vessel segmentation seed points are clearly superiorto other interactive methods avoiding the need to tracedifficult to discern boundaries, drawing very thin scribblesor outlining many bounding boxes. Experimental results ona set of mice dura mater epifluorescence images indicate ourresults are better than other fully automatic or interactivesegmentation methods. We are currently integrating oursemi-automated interactive vessel segmentation method formore rapid ground truth generation and within an automaticimage analytics framework for understanding the influenceof physiological processes on microvasculature morphology.

REFERENCES

[1] A. X. Falcao, J. K. Udupa, S. Samarasekera, S. Sharma, B. E. Hirsch,and R. Lotufo, “User-steered image segmentation paradigms: Livewire and live lane,” Graphical Models and Image Processing, vol. 60,no. 4, pp. 233–260, 1998.

[2] E. N. Mortensen and W. A. Barrett, “Interactive segmentation withintelligent scissors,” Graphical Models and Image Processing, vol.60, no. 5, pp. 349–384, 1998.

[3] A. Perez-Rovira, T. MacGillivray, E. Trucco, K. S. Chin, K. Zutis,C. Lupascu, D. Tegolo, A. Giachetti, P. J. Wilson, A. Doney, andB. Dhillon, “VAMPIRE: Vessel assessment and measurement platformfor images of the retina,” in IEEE EMBC, 2011, pp. 3391–3394.

[4] E. Turetken, F. Benmansour, B. Andres, P. Glowacki, H. Pfister, andP. Fua, “Reconstructing curvilinear networks using path classifiersand integer programming,” IEEE Trans. Patern Analysis and MachineIntelligence, 2016.

[5] S. Meena, V. B. S. Prasath, K. Palaniappan, and G. Seetharaman,“Elastic body spline based image segmentation,” in Proc. IEEE Int.Conf. on Image Processing (ICIP), 2014, pp. 4378–4382.

[6] S. Meena, K. Palaniappan, and G. Seetharaman, “Interactive imagesegmentation using elastic interpolation,” in IEEE Int. Symposium onMultimedia (ISM), 2015, pp. 307–310.

[7] S. Meena, K. Palaniappan, and G. Seetharaman, “User driven sparsepoint based image segmentation,” in IEEE Int. Conf. on ImageProcessing (ICIP), 2016.

[8] C. Kirbas and F. Quek, “A review of vessel extraction techniques andalgorithms,” ACM Computing Surveys, vol. 36, no. 2, pp. 81–121,2004.

[9] Y. Boykov and M.-P. Jolly, “Interactive graph cuts for optimalboundary & region segmentation of objects in N-D images,” in IEEEInternational Conference on Computer Vision, 2001, pp. 105–112.

[10] L. Grady, “Random walks for image segmentation,” IEEE Trans.on Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp.1768–1783, 2006.

[11] V. B. S. Prasath, F. Bunyak, O. Haddad, O. Glinskii, V. Glinskii,V. Huxley, and K. Palaniappan, “Robust filtering based segmentationand analysis of dura mater vessels using epifluorescence microscopy,”in 35th IEEE EMBC, 2013, pp. 6055–6058.

[12] R. Pelapur, V. B. S. Prasath, F. Bunyak, O. V. Glinskii, V. V. Glinsky,V. H. Huxley, and K. Palaniappan, “Multi-focus image fusion onepifluorescence microscopy for robust vascular segmentation,” in 36thIEEE EMBC, 2014, pp. 4735–4738.

[13] V. B. S. Prasath, R. Pelapur, O. V. Glinskii, V. V. Glinsky, V. H.Huxley, and K. Palaniappan, “Multi-scale tensor anisotropic filteringof fluorescence microscopy for denoising microvasculature,” in IEEEInternational Symposium on Biomedical Imaging (ISBI), 2015, pp.540–543.

[14] K. Palaniappan, J. Uhlmann, and D. Li, “Extensor based imageinterpolation,” in IEEE Int. Conf. Image Processing (ICIP), 2003,vol. 2, pp. 945–948.

5916