stellate masses and histologic grades in breast cancer

7/23/2019 Stellate Masses and Histologic Grades in Breast Cancer

1/13

d Original Contribution

STELLATE MASSES AND HISTOLOGIC GRADES IN BREAST CANCER

CHIN-YUANCHANG,*SHOU-JENKUO,y HWA-KOONWU,z YU-LENHUANG,x and DAR-RENCHEN*y

* Cancer Research Center, Changhua Christian Hospital, Changhua, Taiwan;y Comprehensive Breast Cancer Center, ChanghuaChristian Hospital, Changhua, Taiwan; z Department of Medical Imaging, Changhua Christian Hospital, Changhua, Taiwan; and

x Department of Computer Science, Tunghai University, Taichung, Taiwan

(Received29 January2013;revised29 October2013; in final form 4 November2013)

AbstractBreast masses with a radiologic stellate pattern often transform into malignancies, but their tendency tobe of low histologic grade yields a better survival rate compared with tumors with other patterns on mammog-raphy screening. This study was designed to investigate the correlation of histologic grade with stellate features

extracted from the coronal plane of 3-D ultrasound images. A pre-processing method was proposed to facilitatethe extraction of stellate features. Extracted features were statistically measured to derive a set of indices thatquantitatively represent the stellate pattern. These indices then went through a selection procedure to build properdecision trees. The splitting rules of decision trees indicated that stellate tumors are associated with low grade.A set of indices from the lowgrade-associated rules hasthe potentialto represent the stellate feature. Further inves-tigation of the hypoechoic region of peripheral tissue is essential to establishment of a complete discriminatingmodel for tumor grades. (E-mail:[email protected]) 2014 World Federation for Ultrasound in Med-icine & Biology.

Key Words: Breast cancer, Stellate mass, Spiculation, Architectural distortion, 3-D ultrasonography, Computer-aided detection.

INTRODUCTION

When a small mass is observed radiologically within the

breast, some characteristics of its appearance, especially

those that are highly indicative of cancer, are regularly

scrutinized by physicians. In practice, architectural

distortion and spiculation are examples of findings that

have a high malignant potential (DOrsi et al. 2003;

Hong et al. 2005). They are commonly observed in

presentations of breast cancer on mammograms (Baker

et al. 2003; Vyborny et al. 2000) and can be clearly

observed on 3-D sonograms (Rotten et al. 1999).

The appearance of architectural distortion and spic-

ulation has been formally described in the literature, andboth are characterized by a similar stellate pattern. In the

Breast Imaging Reporting and Data System (DOrsi et al.

2003), architectural distortion is described as a condition

in which The normal architecture is distorted with no

definite mass visible. This includes spiculations radiating

from a point, and focal retraction or distortion of the edge

of the parenchyma. For masses having a definite visible

form, spiculation is characterized by spiculated margins

and is described as sharp lines projecting from the

mass. These two formal descriptions of the radiologic

stellate presence implicitly indicate two generally known

phenomena: the pulling of Coopers ligaments into a

neoplasm and the desmoplastic reaction induced by the

invasion of cancer cells. No matter which phenomenon

dominates the process of stellate morphogenesis, a stel-

late pattern in radiographs is commonly recognized as a

manifestation of aggressive cancer (Hong et al. 2005).

On the other hand, several studies have found thesethat the observation of these two stellate patterns on

mammographic screening is a good prognostic factor in

invasive breast cancer. In a study of 212 consecutive

patients with invasive breast carcinoma, De Nunzio

et al. (1997) found a significant correlation between

mammographic spiculation and low histologic grade.

They pointed out that low-grade tumors often provoke a

desmoplastic reaction in the surrounding parenchyma,

whereas the absence or paucity of such a reaction often

leads to the formation of ill-defined and high-grade

Address correspondence to: Dar-Ren Chen, ComprehensiveBreast Cancer Center, Changhua Christian Hospital, 135 NanhsiaoStreet, Changhua 50006, Taiwan. E-mail: [email protected]

Conflicts of interest: The authors have indicated that they have noconflicts of interest regarding the content of this article.

904

Ultrasound in Med. & Biol., Vol. 40, No. 5, pp. 904916, 2014Copyright 2014 World Federation for Ultrasound in Medicine & Biology

Printed in the USA. All rights reserved0301-5629/$ - see front matter

http://dx.doi.org/10.1016/j.ultrasmedbio.2013.11.006
mailto:[email protected]:[email protected]://dx.doi.org/10.1016/j.ultrasmedbio.2013.11.006http://dx.doi.org/10.1016/j.ultrasmedbio.2013.11.006http://dx.doi.org/10.1016/j.ultrasmedbio.2013.11.006http://dx.doi.org/10.1016/j.ultrasmedbio.2013.11.006mailto:[email protected]:[email protected]


2/13

masses. In another prognostic study of 714 women with

1- to 14-mm invasive breast carcinoma, Tabar et al.

(2004) reported that patients with stellate lesions had a

95% 24-y survival rate. A stellate lesion with no associ-

ated calcifications was reported to be the most common

mammographic feature (59%) in their study group, and

survival rates rose to 99% among women with 1- to9-mm stellate lesions. This excellent survival provides

compelling evidence for authors to judge that there is lit-

tle potential for adjuvant therapy or radical surgery to

improve further on the survival. Similar findings were

also observed by Alexander et al. (2006) and reported

in their study, in which they concluded that stellate tu-

mors 114 mm in size had a significantly better survival

rate (98.9%) than tumors with other patterns, even though

there were no differences in treatment. Such a signifi-

cantly better survival rate suggested that the stellate

pattern is an important prognostic factor and that the

effect of adjuvant therapy on these tumors might be small.

Evans et al. (2006)confirmed again that both spiculation

and architectural distortion are associated with a low his-

tologic grade. Although mammographic spiculation is an

independent and good prognostic factor in invasive breast

cancer, as the authors concluded, the mechanism respon-

sible for the beneficial prognostic effect of mammo-

graphic spiculation is not clear.

In short, a mass with a stellate finding often turns out

to be malignant, but there is a strong possibility that the

patient will survive longer. Such a paradoxical associa-

tion was observed to exist in other potentially malignant

findings as well; in fact, high-grade tumors can look like

benign tumors on both mammograms and sonograms

(Lamb et al. 2000). Numerous studies have developed

computer-aided detection (CAD) algorithms to facilitate

the detection of malignant masses (Tang et al. 2009).

However, the stellate pattern is still a common cause of

false-negative findings on screening mammograms,

even with the help of CAD systems (Baker et al. 2003;

Ko et al. 2006; Philpotts 2009). Plenty of CAD studies

on 2-D ultrasonography (US) have obtained promising

results (Cheng et al. 2010), and most of the work was de-

signed to discriminate between benign lesions and malig-

nant tumors. In addition to the paradoxical observationsof radiologists (Alexander et al. 2006; De Nunzio et al.

1997; Evans et al. 2006; Lamb et al. 2000; Tabar et al.

2004), we felt that current CAD techniques lacked

sufficient realization in discriminating high-grade tumors

from low-grade tumors (Chen and Hsiao 2008). There-

fore, analyzing the characteristics of stellate patterns

among malignant tumors can be helpful in reducing false

negatives.

With the emergence of 3-D US, the possibility of

improving the false detections of CAD is increased by

the availability of different perspectives (Fig. 1). In

different anatomic planes, image characteristics of a

tumor are comprehensively revealed, and it is known

that the stellate pattern is observed better on 3-D US cor-

onal sections (Chen and Lai 2011; Cho et al. 2006; Rotten

et al. 1999). This study takes advantage of this

circumstance on 3-D US and examines the possibility

of classifying tumor grades with the stellate feature ex-

tracted from the coronal section. A method to derive spic-

ulation diagrams (Huang et al. 2004) was reproduced at

the first attempt, but the numbers of peaks and valleys

in the spiculation diagram were not clearly differentiable

between all malignant cases (Chang et al. 2012). This

study proposes a mechanism, inspired by the work of

spiculation diagrams, to extract features that are more

representative of the stellate pattern. Through this effec-

tive mechanism, we were able to establish a set of poten-

tial stellate indices that reveal the associativity of all

malignant cases.

Fig. 1. Anatomic planes of 3-D ultrasound images.

Stellate pattern/histologic grade association in breast cancer d C.-Y. CHANGet al. 905


3/13

METHODS

The coronal slices were reconstructed from 3-D

volumetric data, and stellate features were extracted

from these coronal slices using image processing tech-

niques. Histograms from sampling the angular spread of

the features were analyzed statistically, and the numerical

indices derived from the measurement were used for clas-sification against known histologic grades. By evaluating

the performance of the classifications, we could identify a

combination of indices with better discriminating capa-

bility. The next sections detail the mechanism in the order

of the processing steps.

Data acquisition

This retrospective study was approved by the institu-

tional review board of Changhua Christian Hospital,

Taiwan; patient approval or informed consent was not

required for the review of medical records. The data on

191 consecutive patients who were diagnosed with infil-

trating ductal carcinoma or mixed-type carcinomas at

Changhua Christian Hospital between July 2007 and

October 2010 were collected. All 191 patients underwent

sonographic examinations before biopsy or surgery. The

3-D sonograms were obtained using the Voluson 730

US system (GE Healthcare, Zipf, Austria) equipped

with a RSP 6-16 transducer. All examinations were per-

formed with fixed settings: mid-frequency, 0.9-kHz pulse

repetition frequency, and 20.6 gain. The resolutions of

the retrieved images were between 0.013 and 0.029 cm/

pixel. One tumor was collected for each patient; and on

sonograms, each of the 191 tumors measured less than

5 cm in diameter because of the limitations of the trans-

ducer probe.

To reveal a strong stellate pattern, this study used

only the slices reconstructed from the coronal plane of

3-D volumetric data. An experienced physician (S.J.K.)

manually marked the region of interest (ROI) by first

selecting from the 3-D image the centermost slice where

tumor diameter was greatest. He then marked two addi-

tional ROIs, one on either side of the centermost slice.

As recommended by another senior physician (D.R.C.),

who reviewed the reconstructed coronal views and corre-

sponding ROIs, 41 inappropriate cases were eliminated

from the data set because of the likely risk they would un-

reliable results. In 11 of the 41 cases, the tumor occupiedthe whole image or more than 50% of the area, leaving

insufficient space for further information processing of

the surrounding tissue; the remaining 30 cases had areas

with 40%50% visibility and some invisible regions of

tumors out of the margins of the images. After exclusion

of the inappropriate cases, 150 cases remained for

analysis.

Specimens obtained through biopsy or surgery were

tested, and each tumor yielded a series of histopathologic

results; only histologic grades were analyzed for the

purpose of this study. The histologic grade of breast car-

cinoma was assessed according to the Nottingham-

modified Bloom-Richardson grading system (Elston

and Ellis 1991), which sums up scores for tubule forma-

tion, nuclear pleomorphism and mitotic count, and then

stratifies breast tumors into grade I (score 5 35, well

differentiated), grade II (score 67, moderately differen-

tiated) and grade III (score 89, poorly differentiated)

malignancies. Of the 150 tumors studied, 24, 100 and

26 were assessed as histologic grades I, II and III, respec-

tively. Examples of the reconstructed coronal slices are

provided inFigure 2, with one image for each grade.

Image preprocessing

Two different kinds of noise were common in our

data source: speckle noise from the inherent character-

istic of ultrasound imaging, and a synthetic flaw between

discontinuous intensities from the reconstruction of coro-

nal slices. Therefore, noise reduction was desired for

all chosen ROI slices before extraction of the features.

Our approach, originating from the sticks algorithm

(Czerwinski et al. 1999), exploited a combination of

Fig. 2. Examples of reconstructed coronal slices: (a) grade I; (b) grade II; (c) grade III.

906 Ultrasound in Medicine and Biology Volume 40, Number 5, 2014


4/13

morphologic operations with line structuring elements

operating in all feasible orientations to reduce the noise

and enhance the stellate features. Two morphologic oper-

ations, spin-sum dilation and spin-sum erosion, were pro-

posed to accomplish this task. In a discrete n 3 nblock,

there exist (2n 2 orientations that can be properly

defined with a connected line segment of odd length equalto n pixels (Soille 2003). Let 4 and. denote conven-

tional dilation and erosion operations (Soille 2003),

respectively. Dilation and erosion of image Iby line L

structured in orientation q are denoted by I4Lq and

I.Lq. By iterating over all possiblekorientations, where

qk5 (k2 1)p/(2n 2 2) andk5 1, 2,. (2n 2 2), spin-

sum dilation and spin-sum erosion can be defined as

spinsum dilationhX

k

I4Lqk (1)

spin

sum erosionhXk

I.Lqk

(2)

The spin-sum dilation, with a line 5 pixels long, that

is,n 5 5 for example, can proceed by first obtaining the

structuring elements consisting of lines structured in ori-

entations 0, 22.5, 45, 67.5, 90, 112.5, 135 and157.5. The morphologic dilation will apply on the imageiteratively using these eight structuring elements, and

summation of all the iterations is then taken. Likewise,

the same procedure becomes the spin-sum erosion by re-

placing the morphologic dilation with morphologic

erosion. In this study, all chosen images of coronal slices

were pre-processed using the proposed method byapplying spin-sum dilation, followed by spin-sum erosion

iteratively using lines of lengthn 5 [3, 5, 7] pixels. The

intermediate and final images were re-scaled to within

the boundary values of data type for the image, whenever

appropriate. These iterative operations gradually extend

the smooth level on a target image with lines up to 7

pixels that are approximately long enough to match the

spot size of the stellate features.

The proposed enhancing method is called the sum of

line orientation with morphologic operations (SLOMO),

and the effect of SLOMO is illustrated inFigure 3, with

comparison to the results using Gaussian filters.

G x;y 5 e x21y2

2s2 (3)

The conventional Gaussian kernel (eqn [3]) has been

used with many applications to obtain a smooth signal,

but the speckle noise in sonograms is the case that does

not fit well into the normal distribution. Figure 3(c) is

the result using a 13 3 13 Gaussian kernel with

s 5ffiffiffi

2p

, and it is obviously not smooth enough at this

scale. With a 19 3 19 Gaussian kernel and s 5 2ffiffiffi

2p

,

Figure 3(d) indicates that the noise is greatly smoothed

out, but the edges between the brighter and darker regions

are also lost extensively. In Figure 3(b), which is the result

using the proposed method, SLOMO reduces undesirednoise at roughly the same smooth scale as in

Figure 3(d); the contours of the rising and falling regions

are well retained, and the edges between regions are

enhanced as well.

Feature extraction

According to formal descriptions of spiculation and

architectural distortion (DOrsi et al. 2003; Stavros et al.

1995), stellate features can be observed on peripheral

tissue with alternating hyperechoic and hypoechoic

lines that converge toward the central core of the mass.This description suggests that in terms of the gray-scale

image, the region surrounding the ROI of a stellate tumor

should have blocks of alternating high (bright) and low

(dark) intensity. The problem of detecting stellate fea-

tures could thus become one of identifying the fluctuation

of rising and falling intensities. The decision as to

Fig. 3. Demonstration of enhancement using sum of line orientation with morphologic operations (SLOMO) withcomparison to the results using Gaussian filters. (a) Coronal slice before enhancement. (b) SLOMO-enhanced image.(c) Image smoothed using a 13 3 13 Gaussian kernel with s 5

ffiffiffi2

p. (d) Smoothed Image smoothed using a 19 3 19

Gaussian kernel with s 5 2

ffiffiffiffi2:

p



5/13

whether a pixel belongs to a rising or falling region can be

made on the basis of the Laplacian. If the Laplacian is

negative, the pixel is considered to be in the influence re-

gion of rise; if the Laplacian is positive, it is in the influ-

ence region of fall.

Although the Laplacian operator is quite useful in

capturing the occurrence of rising and falling regions, itis also quite sensitive to noise. That is why the pre-

processing stage for noise elimination must occur before

feature detection with the Laplacian. However, even a

minor disagreement over the decision made between

neighboring pixels can still cause a one-piece feature to

fragment. Our approach used a 13 3 13 Laplacian of

Gaussian (LoG) kernel (eqn[4],s5ffiffiffi

2p

) as the detector,

and let the Gaussian kernel smooth out the fluctuation of

decision making between neighboring pixels.

V2Gx;y5 v2Gx;yvx2

1v2Gx;y

vy2 5

x21y222s2

s4 e2

x21y2

2s2

(4)

The stellate features extracted by LoG from the

SLOMO-enhanced image are shown inFigure 4, where

the rising and falling regions are superimposed on the

enhanced image color-coded in orange and green, respec-

tively.Figure 4also illustrates the other three refinements

in which the rising regions are further restricted to inten-

sities larger than the mean, third quartile and mean1

standard deviation (SD); likewise, the falling regions

are further restricted to intensities smaller than the

mean, first quartile, and mean 2 SD. These statistical

quantities are calculated from the SLOMO enhanced im-

age. Some results inFigure 4seem to appear more visu-

ally precise than the others, but it is not easy to identify

the one that is more representative by the eye alone. All

the features extracted at these four different levels were

thus examined in this study: LoG without thresholding,

LoG at the level of the mean, LoG at the level of the quar-

tile and LoG at the level of mean 6 SD. The term level

refers to a specific threshold level that restricts the fea-

tures extracted by LoG.

Fig. 4. Stellate features were extracted at four different levels and in four different ranges. (a) The coronal slice, in thisexample, was first enhanced by sum of line orientation with morphologic operations (SLOMO); stellate features werethen extracted from and superimposed on the enhanced image. (be) Features extracted in full range by LoG withoutthresholding, at the level of the quartile, at the level of mean 6 standard deviation and at the level of the mean, respec-tively. In addition to the full range, (fh) illustrate features extracted in 3, 4 and 5 mm around the region of interest, respec-

tively, using the same level as in (e).



6/13

The techniques discussed so far extract features

from a full available range, that is, the whole image

excluding the ROI. This study also evaluated features

within other ranges as well. The term range refers to a

specific spatial distance around the ROI from which the

features are extracted. Figure 4(eh) illustrates the fea-

tures extracted by LoG in four ranges: full range and 3,4 and 5 mm around the ROI, respectively. To this end,

the features extracted in these four different ranges

were also examined.

Formalization of indices

Extracted features were numerically formalized as

measuring indices before classification. Four numerical

indicesnumber of rising regions (Nr), number of falling

regions (Nf), area ratio of rising regions (ARr) and area

ratio of falling regions (ARf)were calculated immedi-

ately after the feature extraction. Nr and Nfare simply

the amounts of rising and falling regions in a chosenrange. For ARr and ARf, areas are approximated by

counting the number of pixels in the region. LetAidenote

the number of pixels in theith region of rises or falls, and

Arangedenote the amount of pixels in a chosen range; ARrand ARfare defined as

ARr 5XNri5 1

Ai

Arange(5)

ARf5XNf

i5 1

Ai

Arange

(6)

Simply counting the number of regions or deter-

mining the area ratio is certainly not enough to reveal

the pattern. Because the features of a stellate pattern

appear as curving lines that radiate from the margin of a

mass, the spatial distribution of rising and falling regions

extracted by LoG must have some kind of relationship

to the degree of stellate presence in the mass. To capture

the spatial distribution of the stellate pattern, images

were divided into 360 angular segments, where each

segment contains pixels having the same orientation

with respect to the centroid of the ROI. The orientation

of a pixel is determined by the dot product of two vectors:a horizontal vector from the centroid and another vector

from thecentroid to that pixel. Therising andfallingpixels

in each of the angular segments were counted separately,

and two angular histograms were then constructed as the

descriptors of stellate features (Fig. 5). The histograms

were re-scaled to within [0, 1] by the factor of the

maximum area of the segment in 360 orientations to elim-

inating the influence of variance in size between cases.

Six indices were statistically measured from each

histogram: density of the histogram (rrand rf), mean of

the histogram (mrandmf), standard deviation of the histo-

gram (sr and sf), skewness of the histogram (grand gf),

kurtosis of the histogram (kr and kf) and maximum

mean difference of the histogram (dr and df). The sub-

scripts r and f refer to rising and falling histograms,

respectively. LetHq denote the fraction in theqth segment

of either the rising or falling histogram; without loss ofgenerality, these six metrics are defined (with subscripts

r and f omitted) as

r51

360

X360q5 1

f Hq ; where f5(

1; if Hq.0

0; otherwise (7)

m51

360

X360q5 1

Hq (8)

s5ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1

360X360q5 1

Hq2m2vuut

(9)

g5

1

360

X360q5 1

Hq2m3!,

s3 (10)

k5

1

360

X360q5 1

Hq2m4!,

s4 (11)

d5maxHq2m (12)The approach used to sample the angular spread of

the stellate features and construct the angular histograms

is similar to the approach used in the study of mammo-

grams (Huo et al. 1995). The histograms can be shown

to preserve the shapes of stellate patterns (Fig. 5cf),

and the indices derived from measuring the histograms

can be treated as descriptors of the shapes. Each index

alone may tell little about the shape, but a combination

of indices can spot the differences between the shapes

and describe how the shapes differ. For example, ideally,

the distribution of the histogram from a highly stellate

tumor is expected to be more oscillating and uneven;

such a shape is highly likely to have a relatively high stan-dard deviation with a relatively low mean.

Classification

The aim of this study was to realize how malignancy

is related to the degree of sonographic stellate presence of

a mass in breast cancer. To achieve quantitative realiza-

tion of this subject, each feature index had to retain its nu-

merical meaning after classification. One reasonable

choice of classifier is the classification and regression

tree (Breiman et al. 1984), which is a binary recursive



7/13

partitioning procedure that grows trees to a maximal size,

until no further splits are possible. The maximal-sized

tree is then pruned back to the root via the cost-

complexity pruning method. This mechanism produces

a sequence of pruned trees, each of which is a candidate

to be the optimal tree. In this study, the splitting rule of

the trees was based on the measure of impurity, using

Ginis diversity index (Breiman et al. 1984; Raileanu

Fig. 5. This angular histogram was constructed by counting the number of rising or falling pixels in each of the angularsegments. (a) Example illustrating the same features as inFigure 4(c), with a red angular segment at 105. (b) The risingpixels were counted (orange) from the intersections of the rising feature and the angular segment. (cf) Results of re-scaled histograms from the rising feature (orange) and the falling feature (green), where (c) and (d) are from the same

features as inFigure 4(c), and (e) and (f) are from the same features as inFigure 4(h).



8/13

and Stoffel 2004), and the final decision tree was

identified by evaluating the predictive performance of

every tree in the pruning sequence based on leave-one-

out cross-validation.

The feature set for the stellate pattern in this study

contained 256 (4 levels3 4 ranges3 16 indices) indices;

such a large number of features against a small set ofobservations is highly likely to yield an over-fitting result

(Bishop 2006). To obtain a classifying model with good

generalization capability, maintaining the ratio of the

number of training samples to the number of features as

high as possible is considered necessary (Theodoridis

and Koutroumbas 2008). Instead of transforming the

features to some other dimensions, this study chose

sequential forward selection (SFS) (Theodoridis and

Koutroumbas 2008) as a simple and straightforward pro-

cedure to select subsets of features. SFS is a sub-optimal

searching technique that relies on the performance of the

classifier itself. The procedure starts by selecting the

feature with the minimum misclassification rate in 1-D

feature vectors, and then selects again from 2-D vectors

that contain feature elements from the previous selecting

step. This procedure was stopped at the selection of 30-D

vectors, and the best combination was manually selected

by comparing the minimum misclassification rate be-

tween the winners among the 3- to 30-D vectors. The

best combination was fed back to the classification and

regression tree, and the final list of selected features

was counted on the optimal sub-tree.

The known response to learn in this study is the histo-

logic grades, which contain bias in the distribution of quan-

tity; that is, grade II has four times more cases than grade I

and grade III. Thus, the performance of classification was

evaluated in two models: the high-low model and three-one

model. The high-low model classified responses into high

grade (III) or low grade (I, II), and the three-onemodel into

(grade III) and (grade I). Because the three-one model was

a model trained without the acknowledgment of grade II

cases, the number of observations differed between the

two models. Three slices of the image were selected for

each of the 150 cases, so there were 450 observations in

the high-low model. With 100 grade II cases eliminated,

there were 150 observations in the three-one model. After

both models were trained, the possibility of improvingfalse positives and false negatives was also evaluated by

verifying the results of the high-low model with a trained

three-one model. The basic idea is that if a tumor is pre-

dicted to be low grade by a high-low model and grade III

by a three-one model, then the tumor probably is a true

grade III as long as the three-one model is sensitive and

precise enough. The same idea applies to the predictive

high-grade and grade I cases: they are possibly true low-

grade cases if the three-one model has a great specificity

and an excellent negative predictive value.

Furthermore, in the experiments, each SFS proce-

dure selected features from two types of subsets for com-

parison. The first type consisted of the features of all four

levels in a specific combination of ranges, where the com-

binations of ranges were organized into three groups: full

range and 3 mm around the ROI; full range and 4 mm

around the ROI; and full range and 5 mm around theROI. These subsets of features were denoted by R3, R4,

and R5, respectively, and each subset contained 128

feature indices (4 levels 3 2 ranges 3 16 indices). The

second type of subset comprised features of all four

ranges in one of the four levels: LoG without threshold-

ing; LoG at the level of the mean; LoG at the level of

the quartile; and LoG at the level of mean 6 SD. These

subsets of features were denoted LX, LM, LQ and LS,

respectively, and each subset contained 64 feature indices

(1 level 3 4 ranges 3 16 indices).

RESULTS

In general, indices selected from features of the

same type and classified in the same model yielded

similar results. Classifications using features from subsets

of the first type had overall better performance than those

from subsets of the second type in both classification

models, and classifications in the three-one model out-

performed those in the high-low model. High specificity

for low-grade cases was achieved by verifying predictive

high-grade cases with a trained three-one model from the

first type of subset.

Performance of the experiments was evaluated with

respect to sensitivity, specificity, accuracy and the area

under the receiver operating characteristic curve (AUC)

(Hanley and McNeil 1982). The significance (p-value)

of a single AUC was tested with the null hypothesis

that AUC is 0.5, and comparison of two AUCs was tested

against the null hypothesis that there is no difference be-

tween them (Hanley and McNeil 1983). Ninety-five

percent confidence intervals (95% CIs) are provided

along with AUCs in the tables, and standard errors (SE)

andp-values of the AUC are given in this section wher-

ever appropriate. The corresponding indices selected

from the SFS procedures are listed with additional char-

acters indicating the origin of the features. After an un-derline, the letters x, m, q and s denote features

originating from LoG, LoG at the level of the mean,

LoG at the level of the quartile and LoG at the level of

mean 6 SD, respectively. For the second character after

the underline, the digits 0, 3, 4 and 5 denote fea-

tures in full range and 3, 4 and 5 mm around the ROI,

respectively.

Table 1summarizes the results of classifications in

the three-one model. The first three rows of data are re-

sults from subsets of the first type. The AUCs are 0.943



9/13

(SE 5 0.0203, p , 0.0001), 0.913 (SE 5 0.0284,

p , 0.0001) and 0.916 (SE 5 0.0262, p , 0.0001),

respectively, for subsets R3, R4 and R5. The remaining

rows inTable 1are the results from subsets of the second

type: LX, LM, LQ and LS; their AUCs are 0.842

(SE 5 0.0364, p , 0.0001), 0.852 (SE 5 0.0364,

p , 0.0001), 0.867 (SE 5 0.0319, p , 0.0001) and

0.861 (SE 5 0.0328, p , 0.0001), respectively. By

comparing all the results in the three-one model, no single

criterion for evaluating subsets of the second type had a

result that was as good as those for the first type. The sub-set with four indices selected from R3 had a better sensi-

tivity (93.6%) and AUC (0.943); the subset with eight

indices selected from R4 had a better accuracy (94%)

and specificity (91.7%). There were no significant differ-

ence in AUCs between R3 and R4 (p 5 0.2932) or be-

tween R3 and R5 (p 5 0.3574); in fact, R4 and R5

tended not to differ (p 5 0.9262). There were also simi-

larities were between the subsets of the second type

(0.5987 # p # 0.8868). However, differences did exist

in the subsets between the first and second types, with

the most significant difference between R3 and all second

types (p , 0.05).

The scores in the high-low model (Table 2) had over-

all poorer sensitivity, accuracy and AUCs than the scores

in the three-one model. All of the AUCs in the high-lowmodel indicated that the models trained from these sub-

sets were still capable (p , 0.0001), but the subsets

with sensitivities below 60% were not acceptable. The

subset with 10 indices selected from R5 had better

Table 1. Performance of the classifications in three-one model with indices selected from different subsets

Subset* Selected indicesy Sensitivity (TPs, FNs)z Specificity (TNs, FPs) Accuracy AUC (95% CI) z

R3 ARf_x0,mr_x3,kf_q0,gf_q3 93.6% (73, 5) 87.5% (63, 9) 90.7% 0.943 (0.893, 0.974)R4 ARf_x0,kf_x0, Nf_x4,rf_x4,mr_x4,rr_q0,

kf_q0,dr_q492.3% (72, 6) 91.7% (66, 6) 92% 0.913 (0.856, 0.953)

R5 sr_x0, ARf_x0,mr_x5, Nr_m5, dr_m5,ARr_q0,rf_s0

92.3% (72, 6) 87.5% (63, 9) 90% 0.916 (0.859, 0.955)

LX ARf_x0, ARf_x3,rr_x3,mr_x3 87.2% (68, 10) 83.3% (60, 12) 85.3% 0.842 (0.773, 0.896)LM ARr_m0, sr_m0, ARr_m3,rr_m4,gf_m4,

kf_m4, df_m485.9% (67, 11) 86.1% (62, 10) 86% 0.852 (0.785, 0.904)

LQ ARr_q0,mr_q0, Nr_q3,gf_q5,df_q5 89.7% (70, 8) 83.3% (60, 12) 86.7% 0.867 (0.802, 0.917)LS ARr_s0, ARf_s0,kf_s0, Nf_s3,gf_s3,

ARr_s4,gr_s4, ARf_s584.6% (66, 12) 86.1% (62, 10) 85.3% 0.861 (0.795, 0.912)

AUC5 area under the receiver operating characteristic curve; CI 5 confidence interval; FNs 5 false negatives; FPs 5 false positives; TNs 5 truenegatives; TPs 5 true positives.

* R3, R4 and R5 denote subsets of the first type, which contained the features of all levels in full range and one of the ranges at 3, 4 and 5 mm aroundthe ROI. LX, LM, LQ, and LS denote subsets of the second type, which contained the features of all ranges in one of the four levels. The designation ofthese subsets is described under Classification.

y The designation of the indices is described under Formalization of Indices. Two additional characters after the underline are used to indicate theorigin of the features. The four levels are denoted by the letters x, m, q and s; the four ranges are denoted by the digits 0, 3, 4 and 5.For details, see Results.

z AUC and corresponding 95% CI for testing against the hypothesis that the area is 0.5.

Table 2. Performance of the classifications in the high-low model with indices selected from different subsets

Subset* Selected indicesy Sensitivity (TPs, FNs) Specificity (TNs, FPs) Accuracy AUC (95% CI)z

R3 mf_x0,rr_x3,dr_m0, rf_q3, ARf_s0 60.3% (47, 31) 93% (346, 26) 87.3% 0.773 (0.731, 0.811)R4 mf_x0, ARr_m4,rr_m4,mf_m4,mf_q4,gf_q4, ARf_s0 70.5% (55, 23) 94.9% (353, 19) 90.7% 0.842 (0.805, 0.875)R5 rr_x5, Nr_m5,rr_m5, kf_m0, ARr_q0,kf_q0,gf_q5,

kf_q5, Nr_s0,mf_s073.1% (57, 21) 96% (357, 15) 92% 0.814 (0.774, 0.848)

LX Nf_x0, ARr_x0,sf_x0, ARr_x4,rf_x4,rr_x5,rf_x5,mf_x5,kf_x5

52.6% (41, 37) 92.5% (344, 28) 85.6% 0.695 (0.651, 0.738)

LM Nr_m0, ARf_m3, rr_m3, dr_m3, mf_m3, ARf_m5, rr_m5 57.7% (45, 33) 91.9% (342, 30) 86% 0.707 (0.663, 0.749)LQ rf_q0,gr_q3, Nf_q3,rr_q4,df_q4,gr_q5,kr_q5 59% (46, 32) 93.8% (349, 23) 87.8% 0.745 (0.702, 0.785)LS ARf_s0,df_s0,rr_s3,kr_s4,rf_s4,mf_s4, Nf_s5 53.9% (42, 36) 92.2% (343, 29) 85.6% 0.746 (0.704, 0.786)

AUC5 area under the receiver operating characteristic curve; CI 5 confidence interval; FNs 5 false negatives; FPs 5 false positives; TNs 5 truenegatives; TPs 5 true positives.

* R3, R4 and R5 denote subsets of the first type, which contained the features of all levels in full range and one of the ranges at 3, 4 and 5 mm aroundthe ROI. LX, LM, LQ and LS denote subsets of the second type, which contained the features of all ranges in one of the four levels. The designation ofthese subsets is described under Classification.

y The designation of the indices is described under Formalization of Indices. Two additional characters after an underline are used to indicate theorigin of the features. The four levels are denoted by the letters x, m, q and s; the four ranges are denoted by the digits 0, 3, 4 and 5.For details, see Results.

z AUC and corresponding 95% CI for testing against the hypothesis that the area is 0.5.

http://-/?-http://-/?-http://-/?-http://-/?-


10/13

sensitivity (73.1%), specificity (96%) and accuracy

(92%); the subset with seven indices selected from R4

had a better AUC (0.842). No significant difference was

found in AUCs between R3 and R5 (p 5 0.3604) and

between R4 and R5 (p 5 0.5098), but R3 and R4 had a

tendency to differ (p 5 0.0935). The AUCs also revealed

that there were no significant differences between subsets

of the second type, with LQ and LS producing the most

similar pair (p 5 0.9768). Besides R3, differences did

exist in the subsets between the first and second types,

with the most significant difference found between R4

and all of the second types (p , 0.05).

Because subsets of the first type out-performed those

of the second type, experiments to improve the false neg-

atives and false positives in the high-low model were per-

formed with trained three-one models from subsets R3,

R4 and R5. The results indicated that 13, 11 and 9 false

positives were correctly detected for subsets R3, R4 and

R5, respectively (Table 3); the corresponding specificities

were improved to 96.5%, 97.9% and 98.4%. The experi-

ments with false negatives, however, did not work out as

expected. Within the 377, 376 and 378 predictive low-

grade cases (false negatives 1 true negatives), 28 (of

31), 22 (of 23), and 19 (of 21) false negatives were

correctly detected using subsets R3, R4 and R5, respec-

tively. However, 137, 145 and 131 true negatives were

incorrectly classified as grade III, and most of them

(.94%) were true grade II.

Although the three-one model scored better using

subsets R3 or R4, both of the decision models consisted

of a rule likely to split more than 50% of the grade III

cases with a single index, which was reused in successivesplits (0.531 # ARf_x0 , 0.543); this rule limited the

ability to describe the shape properly. Trees of the high-

low models tended to be large with many small branches.

Indices selected from subset R5 in the three-one model

formed a decision model that probably had the best

descriptive capability in all experiments.Figure 6depicts

a portion of the optimal tree from subset R5; each of

the nodes is labeled with its class membership, instead

of the estimated probability, to make it easier to spot

where the larger branch is split. A strong split leading

to a 63.9% (46 out of 72) possibility of grade I cases

can be found by following the route of rules

ARf_x0 $ 0.543, mr_x5 , 0.284, sr_x0 $ 0.082 and

dr_m5 $ 0.35. This combination of rules is consistent

with the shape expected to present on highly stellate

tumors. A 44.9% (35/78) possibility of grade III cases

can be found from ARf_x0 , 0.543, AR

r_q0 , 0.245

and rf_s0 , 0.376; another 30.8% (24/78) possibility

of grade III cases is split by ARf_x0 $ 0.543,

mr_x5 $ 0.284, dr_m5 , 0.589 and sr_x0 , 0.114.

Both combinations of rules are consistent with what is

expected to be the shape of less stellate tumors.

One of the examples that fit the rules well is illus-

trated inFigure 2(a), and can be correctly classified as a

grade I for its ARf_x0 5 0.555, mr_x5 5 0.273,

sr_x0 5 0.114 and dr_m5 5 0.564. The case in

Figure 4is also correctly classified as a grade I through

a different path: ARf_x0 5 0.53, ARr_q0 5 0.195,

rf_s0 5 0.378 and Nr_m5 5 17.Figure 7(a) is a grade

III case fitting the rules ARf_x0 5 0.534, ARr_q0 5

0.208 and rf_s0 5 0.242. Another grade III case in

Figure 2(c) is taken for ARf_x0 5 0.549, mr_x5 5

0.298, dr_m5 5 0.376 and sr_x0 5 0.084. One of the

false negatives from model R5-R5 in Table 3 is illustrated

inFigure 7(c) and is classified as a low grade and grade I,

but actually is a true grade III. All of its relevant indices

are listed for comparison: ARf_x0 50.545, ARr_q0 5

0.197, mr_x5 5 0.276, sr_x0 5 0.084, dr_m5 5 0.39,

rf_s0 5 0.302 and Nr_m5 5 16. Figure 7(e) is one of

the false positives and is classified as both high grade

and grade III, but is a true grade II case, from the

model R5-R5 in Table 3. Its relevant indices areARf_x0 5 0.532, ARr_q0 5 0.181, mr_x5 5 0.314,

sr_x0 5 0.083, dr_m5 5 0.452, rf_s0 5 0.292 and

Nr_m5 5 18.

DISCUSSION

Stellate patterns are highly suspicious for malig-

nancy (Hong et al. 2005), and a high-grade tumor can

look like a benign one (Lamb et al. 2000). These two per-

ceptions can lead to the question: Does a highly stellate

Table 3. Improvements on the false positives in the high-low model

Model*

Original high-grade prediction Improvement after verification

Predictive grade I (high, low)y Predictive grade III (high, low)y False positives True negatives Specificity

R3-R3 13 (0, 13) 60 (47, 13) 13 359 96.5%R4-R4 11 (0, 11) 63 (55, 8) 8 364 97.9%R5-R5 9 (0, 9) 63 (57, 6) 6 366 98.4%

* Rn-Rndenotes that both the high-low model and three-one model use the Rnsubset, as indicated inTables 1and2.y Number of high-grade cases that are classified as grade I or III by a trained three-one model, with numbers of true high-grade and low-grade cases in

parentheses.

http://-/?-http://-/?-


11/13

mass tend to be malignant, but low grade? The answer can

affect decision making on the use of adjuvant therapeutic

regimens. Some studies have answered the question

through visual mammographic assessment (De Nunzio

et al. 1997; Evans et al. 2006). This study found a

correlation using the rules that sonographic feature

indices are split to build the classification trees

(Breiman et al. 1984). Using the observations from the

trees of R3, R4 and R5 of both models, the associated

rules for tumor grades are summarized as follows, with

selected levels and ranges specified in parentheses. The

low histologic grades tend to be associated with the rules:

lower rr (x3, x5, and q0), lower mr (x3, x4, and x5), higher

mf(m4, q4, and s0), highersr(x0) and higherdr(m0, m5,

and q4). These rules represent the shape of a higher oscil-

lating histogram and match the findings that state stellate

tumors are associated with low grade (Alexander et al.

2006; De Nunzio et al. 1997; Evans et al. 2006; Lambet al. 2000; Tabar et al. 2004). Because all the decision

trees in this study are binary trees, high histologic grade

tends to be associated with rules opposite those of the

aforementioned indices: higher rr, higher mr, lower mf,

lower sr and lower dr. These rules represent the shape

of a more uniformly distributed histogram and indicate

that a high-grade tumor tends to be less stellate. The re-

maining indices were excluded from the discussion for

being inconsistent intra- and inter-trees or having little

impact on decision making.

When probabilities and impurities were compared

between split nodes, the links of the high grade-

associated rules seemed to be weaker than those between

low grade and its associated rules. In the high-low model,

especially, the decision rules for high grade usually had

more small splits and larger impurities. We can interpret

this phenomenon as meaning that the less stellate rules

are insufficient to make the decision for high-grade tu-

mors, even though the low grade-associated rules are

found to represent highly stellate shapes. This suggests

that the feature extracting method of this study may be

too specialized to pick up features other than the stellate

ones, and this is believed to be the main reason why the

classifications in the three-one model out-performed

those in the high-low model. Therefore, supplementing

the feature set with finer high-grade features may be the

key to improving the false negatives.

It is worth noting that the aforementioned associatedrules are mostly fromthe rising feature, which is the feature

from the brighter, hyperechoic part of the peripheral tissue.

This indicates that the rising features produced more stable

indices with consistent rules, and it may imply that the ma-

jor contributor to the stellate features was the architectural

distortion in this study. The lack of a stable falling feature

was probably not a coincidence, and the feature set was

also found to be weak in discriminating high-grade cases;

thus, finding the hidden pattern in the hypoechoic region of

peripheral tissue may reveal the high-grade features.

Fig. 6. A portion of the optimal decision tree from R5 of the three-one model, with class membership labeled on eachnode (I 5 grade I, III 5 grade III,n 5 subtotal) and the splitting rules labeled on each edge. The probability of classi-

fication can be estimated from the membership.



12/13

A couple of known limitations need to be addressed

before considering how to improve the system. First, the

number of cases of breast cancer was limited to the avail-

ability of 3-D images and further limited by the 5-cm

capability of the transducer probe for the 3-D ultrasound

system. Second, the number of grade II cases was

unavoidably a lot larger than that of the other grades.

To simplify the feature extracting procedure and increase

the observations, this study made a compromise by using

three slices individually instead of 3-D data as a whole for

each case. A further improvement regarding this issue

should be the primary consideration before deriving a

complete and effective classification model.

In summary, we designed and used a feature extract-

ing method specialized for the presentation of stellate pat-

terns on 3-D ultrasound images The correlation between

Fig. 7. (a) Correctly classified grade III case. (c) Predictive low-grade, but true grade III case. (e) Predictive high-grade,but true grade II case. (b, d, f) Corresponding features with a full range at the mean 6 standard deviation level (b), 5-mm

range at the level of the mean (d) and full range at the quartile level (f).



13/13

histologic grade and stellate pattern in breast cancer was

illustrated through the splitting rules of decision trees,

and once again, stellate tumors were confirmed to be

associated with low grade. The chosen set of indices in

the associated rules has the potential to represent stellate

features. The hypoechoic region deserves further investi-

gation to obtain a proper set of high grade-associatedrules. A more clever approach to circumvent the limita-

tions of the number of cases (Chen et al. 2002) will be un-

dertaken in future work. Two additional classifying

models, three-two (for grades III and II) and two-one

(for grades II and I), should also be considered to work

together with the three-one and high-low models.

AcknowledgmentsEditorial support was provided by Yu-Fen Wang.

REFERENCES

Alexander MC, Yankaskas BC, Biesemier KW. Association of stellatemammographic pattern with survival in small invasive breast

tumors. Am J Roentgenol 2006;187:2937.Baker JA, Rosen EL, Lo JY, Gimenez EI, Walsh R, Soo MS. Computer-

aided detection (CAD) in screening mammography: Sensitivity ofcommercial CAD systems for detecting architectural distortion.Am J Roentgenol 2003;181:10831088.

Bishop CM. Pattern recognition and machine learning. New York:Springer; 2006.

Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification andregression trees. Boca Raton, FL: Chapman and Hall/CRC Press;1984.

Chang CY, Huang YL, Chen DR. Spiculation analysis of breast tumorson 3D ultrasound. In: Proceedings, 2012 International Symposiumon Computer, Consumer and Control (IS3C, 2012), Taichung,Taiwan, 46 June 2012:594597.

Chen DR, Hsiao YH. Computer-aided diagnosis in breast ultrasound.J Med Ultrasound 2008;16:4656.

Chen DR, Kuo WJ, Chang RF, Moon WK, Lee CC. Use of the bootstraptechnique with small training sets for computer-aided diagnosis inbreast ultrasound. Ultrasound Med Biol 2002;28:897902.

Chen DR, Lai HW. Three-dimensional ultrasonography for breastmalignancy detection. Expert Opin Med Diagn 2011;5:253261.

Cheng HD, Shan J, Ju W, Guo Y, Zhang L. Automated breast cancerdetection and classification using ultrasound images: A survey.Pattern Recognition 2010;43:299317.

Cho N, Moon WK, Cha JH, Kim SM, Han BK, Kim EK, Kim MH,Chung SY, Choi HY, Im JG. Differentiating benign from malignantsolid breast masses: comparison of two-dimensional and three-dimensional US. Radiology 2006;240:2632.

Czerwinski RN, Jones DL, OBrien WD Jr. Detection of lines andboundaries in speckle imagesapplication to medical ultrasound.IEEE Trans Med Imaging 1999;18:126136.

DOrsi CJ, Mendelson EB, Ikeda DM. Breast imaging reporting and

data system: ACR BI-RADS-Breast Imaging Atlas. 4th ed. Reston,VA: American College of Radiology; 2003.

De Nunzio MC, Evans AJ, Pinder SE, Davidson I, Wilson ARM,Yeoman LJ, Elston CW, Ellis IO. Correlations between the mammo-graphic features of screen detected invasive breast cancer and path-ological prognostic factors. Breast 1997;6:146149.

Elston CW, Ellis IO. Pathological prognostic factors in breast cancer. I.The value of histological grade in breast cancer: Experience from alarge study with long-term follow-up. Histopathology 1991;19:403410.

Evans AJ, Pinder SE, James JJ, Ellis IO, Cornford E. Is mammographicspiculation an independent, good prognostic factor in screening-detected invasive breast cancer? Am J Roentgenol 2006;187:13771380.

Hanley JA, McNeilBJ. Themeaning anduse of thearea under a receiveroperating characteristic (ROC) curve. Radiology 1982;143:2936.

Hanley JA, McNeil BJ. A method of comparing the areas under receiveroperating characteristic curves derived from the same cases. Radi-ology 1983;148:839843.

Hong AS,Rosen EL,Soo MS,Baker JA. BI-RADS forsonography:Pos-itive and negative predictive values of sonographic features. Am JRoentgenol 2005;184:12601265.

Huang SF, Chang RF, Chen DR, Moon WK. Characterization of spicu-lation on ultrasound lesions. IEEE Trans Med Imaging 2004;23:111121.

Huo Z, Giger ML, Vyborny CJ, Bick U, Lu P, Wolverton DE,Schmidt RA. Analysis of spiculation in the computerized classifica-

tion of mammographic masses. Med Phys 1995;22:15691579.Ko JM, Nicholas MJ, Mendel JB, Slanetz PJ. Prospective assessment of

computer-aided detection in interpretation of screening mammog-raphy. Am J Roentgenol 2006;187:14831491.

Lamb PM, Perry NM, Vinnicombe SJ, Wells CA. Correlation betweenultrasound characteristics, mammographic findings and histologicalgrade in patients with invasive ductal carcinoma of the breast. ClinRadiol 2000;55:4044.

Philpotts LE. Can computer-aided detection be detrimental to mammo-graphic interpretation? Radiology 2009;253:1722.

Raileanu L, Stoffel K. Theoretical comparison between the Giniindex and information gain criteria. Ann Math Artif Intell 2004;41:7793.

Rotten D, Levaillant JM,Zerat L. Analysis of normal breasttissue andofsolid breast masses using three-dimensional ultrasound mammog-raphy. Ultrasound Obstet Gynecol 1999;14:114124.

Soille P. Morphological image analysis: Principles and applications.Secaucus, NJ: Springer-Verlag; 2003.

Stavros AT, Thickman D, Rapp CL, Dennis MA, Parker SH,Sisney GA. Solid breast nodules: Use of sonography to distin-guish between benign and malignant lesions. Radiology 1995;196:123134.

Tabar L, Tony Chen HH, Amy Yen MF, Tot T, Tung TH, Chen LS,Chiu YH, Duffy SW, Smith RA. Mammographic tumor featurescan predict long-term outcomes reliably in women with 114-mminvasive breast carcinoma. Cancer 2004;101:17451759.

TangJ, RangayyanRM, Xu J, El Naqa I, YangY. Computer-aideddetec-tion and diagnosis of breast cancer with mammography: Recentadvances. IEEE Trans Inf Technol Biomed 2009;13:236251.

Theodoridis S, Koutroumbas K. Pattern recognition. Burlington, MA:Academic Press; 2008.

Vyborny CJ, Doi T, OShaughnessy KF, Romsdahl HM, Schneider AC,

Stein AA. Breast cancer: Importance of spiculation in computer-aided detection. Radiology 2000;215:703707.

http://refhub.elsevier.com/S0301-5629(13)01149-6/sref1http://refhub.elsevier.com/S0301-5629(13)01149-6/sref1http://refhub.elsevier.com/S0301-5629(13)01149-6/sref1http://refhub.elsevier.com/S0301-5629(13)01149-6/sref2http://refhub.elsevier.com/S0301-5629(13)01149-6/sref2http://refhub.elsevier.com/S0301-5629(13)01149-6/sref2http://refhub.elsevier.com/S0301-5629(13)01149-6/sref2http://refhub.elsevier.com/S0301-5629(13)01149-6/sref3http://refhub.elsevier.com/S0301-5629(13)01149-6/sref3http://refhub.elsevier.com/S0301-5629(13)01149-6/sref4http://refhub.elsevier.com/S0301-5629(13)01149-6/sref4http://refhub.elsevier.com/S0301-5629(13)01149-6/sref4http://refhub.elsevier.com/S0301-5629(13)01149-6/sref5http://refhub.elsevier.com/S0301-5629(13)01149-6/sref5http://refhub.elsevier.com/S0301-5629(13)01149-6/sref6http://refhub.elsevier.com/S0301-5629(13)01149-6/sref6http://refhub.elsevier.com/S0301-5629(13)01149-6/sref6http://refhub.elsevier.com/S0301-5629(13)01149-6/sref7http://refhub.elsevier.com/S0301-5629(13)01149-6/sref7http://refhub.elsevier.com/S0301-5629(13)01149-6/sref8http://refhub.elsevier.com/S0301-5629(13)01149-6/sref8http://refhub.elsevier.com/S0301-5629(13)01149-6/sref8http://refhub.elsevier.com/S0301-5629(13)01149-6/sref9http://refhub.elsevier.com/S0301-5629(13)01149-6/sref9http://refhub.elsevier.com/S0301-5629(13)01149-6/sref9http://refhub.elsevier.com/S0301-5629(13)01149-6/sref9http://refhub.elsevier.com/S0301-5629(13)01149-6/sref10http://refhub.elsevier.com/S0301-5629(13)01149-6/sref10http://refhub.elsevier.com/S0301-5629(13)01149-6/sref10http://refhub.elsevier.com/S0301-5629(13)01149-6/sref11http://refhub.elsevier.com/S0301-5629(13)01149-6/sref11http://refhub.elsevier.com/S0301-5629(13)01149-6/sref11http://refhub.elsevier.com/S0301-5629(13)01149-6/sref12http://refhub.elsevier.com/S0301-5629(13)01149-6/sref12http://refhub.elsevier.com/S0301-5629(13)01149-6/sref12http://refhub.elsevier.com/S0301-5629(13)01149-6/sref12http://refhub.elsevier.com/S0301-5629(13)01149-6/sref13http://refhub.elsevier.com/S0301-5629(13)01149-6/sref13http://refhub.elsevier.com/S0301-5629(13)01149-6/sref13http://refhub.elsevier.com/S0301-5629(13)01149-6/sref13http://refhub.elsevier.com/S0301-5629(13)01149-6/sref14http://refhub.elsevier.com/S0301-5629(13)01149-6/sref14http://refhub.elsevier.com/S0301-5629(13)01149-6/sref14http://refhub.elsevier.com/S0301-5629(13)01149-6/sref14http://refhub.elsevier.com/S0301-5629(13)01149-6/sref15http://refhub.elsevier.com/S0301-5629(13)01149-6/sref15http://refhub.elsevier.com/S0301-5629(13)01149-6/sref16http://refhub.elsevier.com/S0301-5629(13)01149-6/sref16http://refhub.elsevier.com/S0301-5629(13)01149-6/sref16http://refhub.elsevier.com/S0301-5629(13)01149-6/sref17http://refhub.elsevier.com/S0301-5629(13)01149-6/sref17http://refhub.elsevier.com/S0301-5629(13)01149-6/sref17http://refhub.elsevier.com/S0301-5629(13)01149-6/sref18http://refhub.elsevier.com/S0301-5629(13)01149-6/sref18http://refhub.elsevier.com/S0301-5629(13)01149-6/sref18http://refhub.elsevier.com/S0301-5629(13)01149-6/sref19http://refhub.elsevier.com/S0301-5629(13)01149-6/sref19http://refhub.elsevier.com/S0301-5629(13)01149-6/sref19http://refhub.elsevier.com/S0301-5629(13)01149-6/sref20http://refhub.elsevier.com/S0301-5629(13)01149-6/sref20http://refhub.elsevier.com/S0301-5629(13)01149-6/sref20http://refhub.elsevier.com/S0301-5629(13)01149-6/sref21http://refhub.elsevier.com/S0301-5629(13)01149-6/sref21http://refhub.elsevier.com/S0301-5629(13)01149-6/sref21http://refhub.elsevier.com/S0301-5629(13)01149-6/sref21http://refhub.elsevier.com/S0301-5629(13)01149-6/sref22http://refhub.elsevier.com/S0301-5629(13)01149-6/sref22http://refhub.elsevier.com/S0301-5629(13)01149-6/sref23http://refhub.elsevier.com/S0301-5629(13)01149-6/sref23http://refhub.elsevier.com/S0301-5629(13)01149-6/sref23http://refhub.elsevier.com/S0301-5629(13)01149-6/sref24http://refhub.elsevier.com/S0301-5629(13)01149-6/sref24http://refhub.elsevier.com/S0301-5629(13)01149-6/sref24http://refhub.elsevier.com/S0301-5629(13)01149-6/sref25http://refhub.elsevier.com/S0301-5629(13)01149-6/sref25http://refhub.elsevier.com/S0301-5629(13)01149-6/sref26http://refhub.elsevier.com/S0301-5629(13)01149-6/sref26http://refhub.elsevier.com/S0301-5629(13)01149-6/sref26http://refhub.elsevier.com/S0301-5629(13)01149-6/sref26http://refhub.elsevier.com/S0301-5629(13)01149-6/sref27http://refhub.elsevier.com/S0301-5629(13)01149-6/sref27http://refhub.elsevier.com/S0301-5629(13)01149-6/sref27http://refhub.elsevier.com/S0301-5629(13)01149-6/sref27http://refhub.elsevier.com/S0301-5629(13)01149-6/sref28http://refhub.elsevier.com/S0301-5629(13)01149-6/sref28http://refhub.elsevier.com/S0301-5629(13)01149-6/sref28http://refhub.elsevier.com/S0301-5629(13)01149-6/sref29http://refhub.elsevier.com/S0301-5629(13)01149-6/sref29http://refhub.elsevier.com/S0301-5629(13)01149-6/sref30http://refhub.elsevier.com/S0301-5629(13)01149-6/sref30http://refhub.elsevier.com/S0301-5629(13)01149-6/sref30http://refhub.elsevier.com/S0301-5629(13)01149-6/sref30http://refhub.elsevier.com/S0301-5629(13)01149-6/sref30http://refhub.elsevier.com/S0301-5629(13)01149-6/sref30http://refhub.elsevier.com/S0301-5629(13)01149-6/sref29http://refhub.elsevier.com/S0301-5629(13)01149-6/sref29http://refhub.elsevier.com/S0301-5629(13)01149-6/sref28http://refhub.elsevier.com/S0301-5629(13)01149-6/sref28http://refhub.elsevier.com/S0301-5629(13)01149-6/sref28http://refhub.elsevier.com/S0301-5629(13)01149-6/sref27http://refhub.elsevier.com/S0301-5629(13)01149-6/sref27http://refhub.elsevier.com/S0301-5629(13)01149-6/sref27http://refhub.elsevier.com/S0301-5629(13)01149-6/sref27http://refhub.elsevier.com/S0301-5629(13)01149-6/sref26http://refhub.elsevier.com/S0301-5629(13)01149-6/sref26http://refhub.elsevier.com/S0301-5629(13)01149-6/sref26http://refhub.elsevier.com/S0301-5629(13)01149-6/sref26http://refhub.elsevier.com/S0301-5629(13)01149-6/sref25http://refhub.elsevier.com/S0301-5629(13)01149-6/sref25http://refhub.elsevier.com/S0301-5629(13)01149-6/sref24http://refhub.elsevier.com/S0301-5629(13)01149-6/sref24http://refhub.elsevier.com/S0301-5629(13)01149-6/sref24http://refhub.elsevier.com/S0301-5629(13)01149-6/sref23http://refhub.elsevier.com/S0301-5629(13)01149-6/sref23http://refhub.elsevier.com/S0301-5629(13)01149-6/sref23http://refhub.elsevier.com/S0301-5629(13)01149-6/sref22http://refhub.elsevier.com/S0301-5629(13)01149-6/sref22http://refhub.elsevier.com/S0301-5629(13)01149-6/sref21http://refhub.elsevier.com/S0301-5629(13)01149-6/sref21http://refhub.elsevier.com/S0301-5629(13)01149-6/sref21http://refhub.elsevier.com/S0301-5629(13)01149-6/sref21http://refhub.elsevier.com/S0301-5629(13)01149-6/sref20http://refhub.elsevier.com/S0301-5629(13)01149-6/sref20http://refhub.elsevier.com/S0301-5629(13)01149-6/sref20http://refhub.elsevier.com/S0301-5629(13)01149-6/sref19http://refhub.elsevier.com/S0301-5629(13)01149-6/sref19http://refhub.elsevier.com/S0301-5629(13)01149-6/sref19http://refhub.elsevier.com/S0301-5629(13)01149-6/sref18http://refhub.elsevier.com/S0301-5629(13)01149-6/sref18http://refhub.elsevier.com/S0301-5629(13)01149-6/sref18http://refhub.elsevier.com/S0301-5629(13)01149-6/sref17http://refhub.elsevier.com/S0301-5629(13)01149-6/sref17http://refhub.elsevier.com/S0301-5629(13)01149-6/sref17http://refhub.elsevier.com/S0301-5629(13)01149-6/sref16http://refhub.elsevier.com/S0301-5629(13)01149-6/sref16http://refhub.elsevier.com/S0301-5629(13)01149-6/sref16http://refhub.elsevier.com/S0301-5629(13)01149-6/sref15http://refhub.elsevier.com/S0301-5629(13)01149-6/sref15http://refhub.elsevier.com/S0301-5629(13)01149-6/sref14http://refhub.elsevier.com/S0301-5629(13)01149-6/sref14http://refhub.elsevier.com/S0301-5629(13)01149-6/sref14http://refhub.elsevier.com/S0301-5629(13)01149-6/sref14http://refhub.elsevier.com/S0301-5629(13)01149-6/sref13http://refhub.elsevier.com/S0301-5629(13)01149-6/sref13http://refhub.elsevier.com/S0301-5629(13)01149-6/sref13http://refhub.elsevier.com/S0301-5629(13)01149-6/sref13http://refhub.elsevier.com/S0301-5629(13)01149-6/sref12http://refhub.elsevier.com/S0301-5629(13)01149-6/sref12http://refhub.elsevier.com/S0301-5629(13)01149-6/sref12http://refhub.elsevier.com/S0301-5629(13)01149-6/sref12http://refhub.elsevier.com/S0301-5629(13)01149-6/sref11http://refhub.elsevier.com/S0301-5629(13)01149-6/sref11http://refhub.elsevier.com/S0301-5629(13)01149-6/sref11http://refhub.elsevier.com/S0301-5629(13)01149-6/sref10http://refhub.elsevier.com/S0301-5629(13)01149-6/sref10http://refhub.elsevier.com/S0301-5629(13)01149-6/sref10http://refhub.elsevier.com/S0301-5629(13)01149-6/sref9http://refhub.elsevier.com/S0301-5629(13)01149-6/sref9http://refhub.elsevier.com/S0301-5629(13)01149-6/sref9http://refhub.elsevier.com/S0301-5629(13)01149-6/sref9http://refhub.elsevier.com/S0301-5629(13)01149-6/sref8http://refhub.elsevier.com/S0301-5629(13)01149-6/sref8http://refhub.elsevier.com/S0301-5629(13)01149-6/sref8http://refhub.elsevier.com/S0301-5629(13)01149-6/sref7http://refhub.elsevier.com/S0301-5629(13)01149-6/sref7http://refhub.elsevier.com/S0301-5629(13)01149-6/sref6http://refhub.elsevier.com/S0301-5629(13)01149-6/sref6http://refhub.elsevier.com/S0301-5629(13)01149-6/sref6http://refhub.elsevier.com/S0301-5629(13)01149-6/sref5http://refhub.elsevier.com/S0301-5629(13)01149-6/sref5http://refhub.elsevier.com/S0301-5629(13)01149-6/sref4http://refhub.elsevier.com/S0301-5629(13)01149-6/sref4http://refhub.elsevier.com/S0301-5629(13)01149-6/sref4http://refhub.elsevier.com/S0301-5629(13)01149-6/sref3http://refhub.elsevier.com/S0301-5629(13)01149-6/sref3http://refhub.elsevier.com/S0301-5629(13)01149-6/sref2http://refhub.elsevier.com/S0301-5629(13)01149-6/sref2http://refhub.elsevier.com/S0301-5629(13)01149-6/sref2http://refhub.elsevier.com/S0301-5629(13)01149-6/sref2http://refhub.elsevier.com/S0301-5629(13)01149-6/sref1http://refhub.elsevier.com/S0301-5629(13)01149-6/sref1http://refhub.elsevier.com/S0301-5629(13)01149-6/sref1

stellate masses and histologic grades in breast cancer

Documents