1

2

A HIERARCHICAL RULE-BASED METHOD FORIMAGE SEGMENTATION USING MAXIMUM GRADIENT PROFILES

A.C.F.Colchester1

R. T. Ritchings2

N.D.Kodikara2

Department of Neurology, Guy's Hospital, London Bridge SElDepartment of Computation, UMIST, PO Box 88,Manchester M60

9RT1QD

For each p i x e l in an image wedetermined the local direct ion ofgreatest grey l e v e l change andlinked pixels in this direction toform maximum gradient prof i l e s(MGP). Adjacent MGP's were linkedside-by-side t o f o r m e d g esections. These edge s e c t i o n scorresponded very closely to thosefrom a Canny operator.

For the present application, whichwas concerned with detection andcharacterisation of blood vesse l son angiograms, subsequent groupingwas res tr ic ted to anti-symmetricedge sections which formed a ridgewith high grey v a l u e s in thecentre. The ridges correspondedclosely with blood ves se l s shownin the image. The width of eachridge and the locus of i t s centre-line were d e r i v e d from a n t i -symmetric MGP p a i r s a n dcorresponded very well with expertjudgement of v e s s e l width andcentre line location. Advantages ofthe MGP grouping method includedexplicit r epresenta t ion of theprofiles of grey values normal toedges; ease of implementation ofhigher-order grouping stages whichwere natural extensions of lowerlevel processes; and hierarchicaldata structure representing imagefeatures at a l l s ca l e s , allowingimproved interaction of bottom-upand top-down processing.

Introduction

Our previous work on knowledgebased interpretat ion of angiogra-phic i m a g e s 1 ' 2 ' 3 has underlinedthe n e e d t o e s t a b l i s h ahierarchical data s t r u c t u r e whichrepresents the image at mul t ip lescales and which allows simplifiedcontrol of the interaction betweenimage-driven and knowledge-drivenprocesses. The method has beendeveloped a s a p o t e n t i a lal ternat ive to the current m u l t i -resolution techniques which useimage b l u r r i n g of d i f f e r e n tdegrees in t h e a n a l y s i s offeatures at multiple scales 4 . Wehave developed the method in thecontext of the ana lys i s of bloodvessel images but i t i s notres t r ic ted to angiograms and, aswill be seen , very few of theprocessing s t e p s use d o m a i n -specific knowledge.

We consider an image as a 3-D greylevel landscape with topologica lfeatures-* and we t r e a t an edge asa " h i l l s i d e " or s teep region onthis 3-D s u r f a c e . The mostfundamental component of the edgeis the p r o f i l e of grey l e v e lchange along l i n e s of maximumgradient. A d j a c e n t l i n e s ofmaximum gradient on an edge wi l lhave s imi l a r p r o f i l e s and tend tobe approximately p a r a l l e l . Thedirections of t h e s e maximumgradient p r o f i l e s (MGP's) a r enormal to the d i r e c t i o n of theedge. Our method of ex t r ac t ingimage structure is based on MGP's,is easy to implement, and has anumber of important properties.

211

AVC 1988 doi:10.5244/C.2.33

Figure 1 Region of interest beforedigitisation

Method

Conventional angiographic imagesshowing the c a r o t i d a r t e r i e s andthe i r branches in the neck wereanalysed. They had a l ready beenphotographically s u b t r a c t e d i . e .radiographic film obtained beforeinjection of c o n t r a s t medium hadbeen subtracted from a film takenimmediately a f t e r i n j e c t i o n .Regions of i n t e r e s t on t h eangiograms were d i g i t i s e d t o128x128x6 b i t r e s o l u t i o n .Examples are shown in Figs 1 and8. The images were t r a n s f e r r e dto an Apollo DN 3000 works ta t ionin the Department of Computationat UMIST where the sys tem wasimplemented.

A pa i r of 3 x 3 Sobel ope ra to r swas used to e s t a b l i s h the loca ldirect ion and magnitude of maximumgradient for e v e r y p i x e l . Athreshold for these g r a d i e n t s wasderived automatically as follows.The image was divided in to non-overlapping 10x10 p ixe l squaresand the Sobel operator magnitudes(regardless of d i r e c t i o n ) wereexamined within every square. The

square with the minimum standarddeviation of Sobel magnitudes('s')was used for c a l c u l a t i n g thethreshold which was then appl iedglobally. All p i x e l s whose Sobeloutput was g r e a t e r than 3s wereretained. The neighbours of eachof these pixels were examined, andany whose Sobel gradient magnitudewas g r e a t e r than 2s were a l s oretained. O the r p i x e l s wereignored.

Subsequent p r o c e s s i n g s t e p sinvolved s u c c e s s i v e l y g roup ingelements to form l a r g e r e lements ,larger elements into s t i l l largerelements, and so on. Pixels witha l o c a l g r a d i e n t above t h ethreshold were l inked along thepath of maximum grad ien t to formMGP's provided tha t the t r ack ofan MGP did not change d i r e c t i o n bymore than 45° in t o t a l . MGP'swere te rmina ted if there were nogradients above th resho ld or if afurther l i n k would i n v o l v e achange of d i r e c t i o n of more than45°. The o v e r a l l d i r e c t i o n (in x &y) of the MGP was obtained from aleas t squares s t ra igh t l ine f i t tothe (x,y) value of the MGP. At

212

the ends of each MGP neighbouringMGP's with the neares t end po in t swere examined, and if the MGP'swere approximately co-l inear theywere l inked end to end to form acomposite MGP. The p o i n t ( s ) ofsteepest s lope of the MGP werefound. MGP's whose s t e e p e s tslope points were near neighbourswere compared, and where theseMGP's had s i m i l a r d i r e c t i o n andmaximum g r a d i e n t t h e y weregrouped to form a s ide by s ide se tof MGP's or "h i l l s ide" region witha normal given by the mean of theMGP direc t ions . Edge sections wereformed from the po in t s of s t e epes tslope on adjacent MGP's by meansof a ru l e se t which (a) a t temptedto m a i n t a i n the edge s e c t i o ndirection p e r p e n d i c u l a r to theMGP's and (b) min imised sma l ldeviations in the direct ion of theedge section (similar to a 5-pointsmoothing). Where appropriate therule se t chose between competingsteepest-slope po in t s on a s ing leMGP or i n t e r p o l a t e d new p o i n t s .The edge sec t ions thus derivedwere compared with edge s ec t i onsderived from Canny and from Marr-Hildreth o p e r a t o r s . The s c a l eparameter of these was se t at 11pixels after some experimentation.

Processing using the edge sectionsderived from the MGP groupingmethod was continued by l ink ingnear-neighbour anti-symmetric edgesections t o form r i d g e s orvalleys. If these were long inextent they corresponded to l i nefeatures in the image. In thepresent a p p l i c a t i o n , where theobjective was to d e t e c t bloodvessels on g o o d q u a l i t yangiographic images on which bloodvessels appeared as r idyes on the3-D sur face , edges without a n t i -symmetric partners forming a ridgewere suppressed. Along the lengthof r idges the width of the r idgeand the locus of i t s c e n t r e - l i n eor symmetric axis were derivedfrom the edge p o i n t s of a n t i -symmetric MGP pa i rs .

The major i ty of the code was inthe form of c o n d i t i o n a l r u l e sprogrammed in L i s p ; the Sobeloperator was programmed in Pascal.

Figure 2A Angio 1: Perspective viewof Maximum-Gradient Profiles shown asi f superimposed on the 3-D grey levelsurface

Results

Two e x a m ples of the digitisedregion of interest are shown inFigures 1 and 8. Figs 2 to 5 showstages of processing of Angiogram1, Figs 9 to 12 stages ofAngiogram 2. Perspective views ofthe extracted MGP's, shown as ifsuperimposed on the 3-D grey levelsurface of the images, are shownin Figures 2 and 9. Fig 2B is anenlarged view of a region justabove the centre of Fig 2A. Whenthe edge points are displayed themethod can be seen to function asa very effective low level edgedetector (Figures 3 and 10). Withfurther processing to extend edgesections and remove those sectionswithout an anti-symmetric partnerforming a ridge, a reliable bloodvessel boundary map was generated(Figures 4 and 11). The detectedvessel centre lines, whichcorresponded very well with anexpert's judgement of centre lineposition, are shown in Figures 5and 12.

213

Figure 2B Angio 1: Enlarged view of MaximumGradient Profiles from region near centre of

Figure 2A

214

Edge sect ions detected by the MGPgrouping method (Figs. 3 and 10)were compared with the Canny (Figs6 and 13)and Marr-Hildreth (Figs 7and 14) o p e r a t o r s . The Marr-Hildreth method suffers from majordistort ion at sharp corners . TheMGP grouping method performscomparably to the Canny (compareFigs 3 with 6 and 10 with 13).

Discussion

Conventional image pre-processingtypically involves the convolutionof simple kernels with image greydata and then thresholding togenerate edge sections. These then

• '• ' : : ' . • ' .

; j . " i

! =1

: ':_ :

. :. ' ":: •• : ; • . :

: : : : : :

i:: ': I '. ::

'•• U [ ̂, . « • : : . • • • : . . ' : : : : :

^ '1/ ;: 1 =•..". :"' ;:

.'• i

i::

•

•j: .! •' : : i •' :

: ..•••«•' i i : ::

• ' ! ! ! ! • • ' ' ': •' • • • '

Figure 4 Angio 1: Blood vessel boundarymap derived from MGP grouping method

I I . • • " « ' ,I I'" I",

'

1 ••».•.•••• «.• • *

L J . ! f •'- ' I•Is •. ! ! • ; • ! 'j 1ii! !

,'l 'l ', J, v ' ! . 1 ' .

I " L. ••••• I I *•! '.••. —•• — iT"•. '. I l l I ' . I

. ••. ti •„• :• i i i i i i •«

" " H ^ ^ A B A • * • « * *

Figure 5 Angio 1: Blood vessel centre-lines derived from MGP grouping method

Figure 3 Angio 1: steepest-slope pointson the Maximum Gradient Profiles

215

—I, I I M

," « ...

^ A, /I ;i

I1 i1 r V•Hii i1

,.W4) J ; ••

nFigure 6 Angio 1: Canny operator out-put with width = 11 pixels

::' v1 '..•. '••! I V : :- :•••,/

fcJ:WJ N H

Figure 7 Angio 1: Marr-Hildrethoperator output with width = 11 pixels

Figure 8 Angio 2: Region of interestbefore digitisation

216

1 \ : ••..•••

i " - i i 'j ; l'

: j !

: i •

I • • • ' : i . . . :

I ...•••• 1

Ij j

!. 1! i

I

t

jI :

i

• :•_: \! t

••:'', " j

. • '

: ' t .

:' l "• _ l .

••:••..

.* : :

:' 1 :"•

:̂

::':I:i

1

:_

i

t

) ']

*•

i •

ii !: t !

I

Figure 9 Angio 2: Perspective view ofMaximum-Gradient Profiles shown as ifsuperimposed on the 3-D grey level surface Figure 11 Angio 2: Blood vessel bound-

ary map derived from MGP grouping method

• - * ' . ~i '•

• : - • , - , •

I,11

~\

ff1 V"'1-1

iLI1

• '. •• L

••••;•••]i

•' •« 11 l i b

ikJ,,

\* 1• ]

Ji| "n

1

1 '•••• i

•1i '<. •«

II r •—ji i

• i

:.,.. -..-—.„

I

1,

Figure 10 Angio 2: Steepest-slopepoints of the Maximum Gradient Profiles

Figure 12 Angio 2: Blood vesselcentre lines derived from MGP groupingmethod

217

—r\ \ \ \ '. '.,, r I -!« -.1 1 :

Figure 13 Angio 2: Canny operator out-put with width = 11 pixels Figure 14 Angio 2: Marr-Hildreth

operator output with width = 11 pixels

have t o be s u b j e c t e d t o adifferent type of processing^ forlinking, s ke 1 e ton i sa t i on , e t c .Our approach also requ i res thesame type of geometric l inkingprocesses to group and extend edgesections i n t o l a r g e r edgesections, l a r g e r edge s e c t i o n sinto objects e t c . , but these sameprocesses are also used at lowerlevels to generate edge sec t i ons .The d i s t i n c t i o n between p r e -processing and subsequent analysisis thereby blurred. S imi l a r ly ,the same basic data s t r u c t u r e swhich are used for h i g h - l e v e l ,more abs t r ac t r ep resen ta t ions ofobjects can be used throughout theprocessing r o u t i n e . In t h e s eways, feedback to ref ine e a r l i e rprocessing r e s u l t s can becontrolled in a more un i fo rmmanner and less re -process ing ofimage grey-level data is required.

With our method, each s t age ofprocessing groups smaller elementsinto larger features. In that thesame bas i c grouping r u l e s areapplied at each s t a g e , and thenumber of fea tures progress ive lydecreases, our method is a type ofpyramidal algorithm^'^ with rapid

convergence. However, u n l i k eRosenfeld's a lgor i thms , the sca leof the elements being linked ateach stage is not predetermined,but is set by local data. As withother pyramidal algorithms, scaleincreases as one ascends from thelowest level to the highest in ourdata s t ructure, but the level of afeature in the hierarchy specifiesthe number of g e n e r a t i o n s ofsubfeatures below i t , r a the r thanabsolute s i z e .

The d i f f e ren t approaches used inthe MGP g r o u p i n g method andconventional edge de t ec to r s havemade i t d i f f i c u l t to decide whichstages of the MGP grouping methodto include in a comparison withmore c o n v e n t i o n a l o p e r a t o r s .Objective comparison was also made

di f f icu l t because of user input tothe processing. To obtain optimumresul ts from the Canny and MarrHildreth o p e r a t o r s r e q u i r e dseveral experiments with differentscale p a r a m e t e r s us ing expe r tknowledge to assess the r e s u l t sbefore the optimum s e t t i n g forthis p a r t i c u l a r app l i ca t ion wasestablished. Our p r e l i m i n a r yevaluation of the MGP grouping

218

method indicates that when appliedto angiographic subtraction imagesi t performs similarly to the Cannyoperator in the de tec t ion of edgefragments. A s i g n i f i c a n tadvantage of the MGP groupingmethod arose in the implementationof l a t e r s t a g e s of p r o c e s s i n g .The linking of anti-symmetric edgesections and d e r i v a t i o n of thecentre-l ine and width of r idgeswere a l l natural extensions of theear l ie r grouping s tages and verysuccessful in operation.

The images so far examined haverela t ively strong fea tures and wehave yet to e v a l u a t e the MGPgrouping method in very noisyimages. Future extensions of theMGP grouping method should help toensure good performance in noisyimages. F i r s t l y , small peaks orridges due to noise , on a la rger"hi l l s ide" represent ing the edgeof a vesse l , wi l l be categor isedas subparts of the la rger edgeand will not prevent detection andcharacterisation of t he l a r g e rstructure. Secondly, i f l o c a llinks are not es tab l i shed at anylevel of t he pyramid a t anylocation, then grouping wi l l beattempted over a wider area. Thisis the si tuation which will oftenarise in noisy images. Processingtime wi l l be increased, but wherestrong local s t r uc tu r e is found,this time penalty will be avoided.

In many types of image, the shapeof the grey-level profiles normalto edges c a r r i e s i m p o r t a n tinformation9 which i s p o t e n t i a l l yuseful both during image-drivensegmentation and d u r i n g imageinterpretat ion u s i n g d o m a i nspecific knowledge . This i spart icularly t rue in q u a n t i t a t i v eangiography, where many of thecalculations about vessel width,cross-sectional a r e a , c r o s s -sectional s h a p e and o t h e rparameters are derived from theprofile of grey level values alonga track perpendicular to the longaxis of the v e s s e l ^ . Traditiona-l ly the c o n s t r u c t i o n of thesetransverse densi ty p ro f i l e s hasrequired several i n i t i a l s t eps :vessel recognit ion (usually withextensive user interact ion); edgelocalisationj centre-l ine loca l i s -

ation; ca lcu la t ion of the normalto the centre- l ine; and selectionof nearest p ixe ls for grey levelsampling. The MGP grouping methoddescribed here p rov ides a muchmore e f f i c i e n t p r o c e s s i n gsequence.

There are s t i l l many aspects ofthe MGP grouping method to beexplored and e v a l u a t e d . Thepreliminary resul ts described herehave been sufficiently encouragingto j u s t i f y further inves t iga t ionand our work in t h i s a rea i scontinuing. Future e x t e n s i o n swill inc lude the d e t e c t i o n andclassif icat ion o f o t h e rtopological f e a t u r e s such asvalleys and grey-level maxima andminima at d i f fe ren t sca les ( h i l ltops and valley bottoms). In th isway the whole image w i l l berepresented without using domain-specific k n o w l e d g e a s ahierarchical t ree s t ruc tu r e whichcodes the r e l a t i o n s h i p betweenfeatures at multiple scales .

References

1. Ri tchings RT and C o l c h e s t e rACF. (1986) D e t e c t i o n ofabnormal i t i e s on c a r o t i dangiograms u s i n g s y n t a c t i ctechniques . P a t t e r nRecognition L e t t e r s 4 3 6 7 -374.

2. Ri tchings RT, C o l c h e s t e r ACFand Wang H - Q . ( 1 9 8 6 )Knowledge b a s e d a n a l y s i sof c a r o t i d a r t e r i o g r a m s .Image and Vis ion Computing 3^217-222. ~

3. Wang H-Q, R i t c h i n g s RT andColchester ACF. (1987) Animage u n d e r s t a n d i n g sys temfor c a r o t i d a n g i o g r a m s .Image and Vis ion Computing 579-84. ~

4. Pizer SM, O l i v e r WR, andBloomberg S H . ( 1 9 8 7 )Hie ra rch ica l shape d e s c r i p -t ion via the m u l t i - r e s o l u t i o nsymmetric a x i s t r a n s f o r m s .Pa t t e rn A n a l y s i s & MachineI n t e l l i g e n c e PAMI-9 505-511.

219

5. Koenderink JJ (1988) Images t ruc tu re . In Viergever MAand Todd-Pokropek A (eds)Mathematics and C o m p u t e rScience in Medical Imaging,Berlin: Springer, 67-104.

6. Shirai Y (1973) A c o n t e x tsens i t ive l ine - f inder . Ar t i -f i c i a l In te l l igence^: 95-119

7. Baugher S and Rosen fe ld A(1986) Boundary loca l i sa t ionin an image pyramid. Pat ternRecognition J^9: 373-395.

10

Rosenfeld A (1987) Pyramidalgorithms fo r e f f i c i e n tvis ion, TR 1866, Center forAutomation R e s e a r c h , Univ.Maryland, College Park, June1987.

Sleigh AC ( 1 9 8 5 ) Theextrac t ion of b o u n d a r i e susing loca l measures dr ivenby r u l e s . BPRA 3 r dIn terna t iona l Conference, St.Andrew's, Scot land .

Colchester ACF (1985) Theeffect of changing Pa CO? oncerebral a r t e r y c a l i b r eestimated by a new techniqueof d y n a m i c q u a n t i t a t i v ed i g i t a l a n g i o g r a p h y . PhDThesis, University of London.

220