ter haar romeny, icpr 2010 mr slice hartcoronair scale toppoints graph theory edge focusing
TRANSCRIPT
ter Haar Romeny, ICPR 2010
MR slice hartcoronair
scale
• toppoints
• graphtheory
Edge focusing
ter Haar Romeny, ICPR 2010
Structures exist at their own scale:
Original = e0 px = e1 px = e2 px = e3 px
Noise edges
ter Haar Romeny, ICPR 2010
100 200 300 400 500
4
2
2
4
6
8
10
The graph of the sign-change of thefirst derivative of a signal as a function of scale is denoted the scale-space signature of the signal.
Zero-crossingsof the secondorder derivative= max of firstorder derivative,
as a functionof scale
ter Haar Romeny, ICPR 2010
The notion of longevity can be viewed of a measure of importance for singularities [Witkin83]. The semantical notions of prominence and conspicuity now get a clear meaning in scale-space theory.
In a scale-space we see the emergence of the hierarchy of structures. Positive and negative edges come together and annihilate in singularity points.
ter Haar Romeny, ICPR 2010
Example:
Lysosomesegmentationin noisy 2-photonmicroscopy3D images ofmacrophages.
ter Haar Romeny, ICPR 2010
Marching-cubes isophote surface ofthe macrophage.
slice 24 slice 21 slice 25 slice 18 slice 22 slice 21
slice 24 slice 23 slice 24 slice 20 slice 18 slice 24
Preprocessing:- Blur with = 3 px- Detect N strongest maxima
ter Haar Romeny, ICPR 2010
We interpolatewith cubic splinesinterpolation35 radial tracksin 35 3Dorientations
ter Haar Romeny, ICPR 2010
The profiles are extremely noisy:
Observation: visually we can reasonably point the steepest edgepoints.
ter Haar Romeny, ICPR 2010
Edge focusingover all profiles.
Choose a startlevel based onthe task, i.e. finda single edge.
ter Haar Romeny, ICPR 2010
Detected 3D points per maximum.
We need a 3D shape fit function.
ter Haar Romeny, ICPR 2010
1
2
,1
23
2 sin, 1
23
cos,
1
23
2 sin, 1
42 15
2 sin2,
1
215
2 cossin, 1
45
3 cos2 1,
1
215
2 cossin, 1
42 15
2 sin2
The 3D points are least square fit with 3D spherical harmonics:
ter Haar Romeny, ICPR 2010
Resulting detection:
ter Haar Romeny, ICPR 2010
An efficient way to detect maxima and saddlepoints is found inthe theory of vector field analysis (Stoke’s theorem)
ter Haar Romeny, ICPR 2010
Topological winding numbers
p Li1dLi2. . .dLin i1i2...inL jpL jpn2p L1dL2 L2dL1
L12 L22
N-D
2-D
is the wedge product (outer product for functionals)
ter Haar Romeny, ICPR 2010
In 2D: the surrounding of the point P is a closed path around P.
The winding number of a point P is defined as the number of times the image gradient vector rotates over 2 when we walk over a closed path around P.
maximum: = 1minumum: = 1regular point: = 0saddle point: = -1monkey saddle: = -2
ter Haar Romeny, ICPR 2010
The notion of scale appears in the size of the path.
Winding number = +1 extremum
Winding number = -1 saddle
ter Haar Romeny, ICPR 2010
Generalised saddle point (5th order): (x+i y)5
The winding numbers add within a closed contour, e.g.A saddle point (-1) and an extremum (+1) cancel, i.e. annihilate.
Catastrophe theory
Winding number = - 4 monkey saddle
ter Haar Romeny, ICPR 2010
ter Haar Romeny, ICPR 2010
2 4 6 8 1010200
300
500
700
1000
Slopefor MR image: 1.66555
2 4 6 8 1010
1000
1500
2000
Slopefor white noise: 1.91549
The number of extrema and saddlepoints decrease as e-N over scale
Decrease of structure over scale scales with the dimensionality.
ter Haar Romeny, ICPR 2010
Fertility Prospects
In most developed countries a postponement of childbearing is seen.
E.g. in the Netherlands: Average age of bearing first child is 30 years.
Computer-Assisted Human Follicle Analysis for Fertility Prospects with
3D Ultrasound
ter Haar Romeny et al., IPMI 1999
Application:
ter Haar Romeny, ICPR 2010
pelvis
oviduct
ovary
uterus
rectum
vagina
anusbladder
vulvaureter
clitoris
Femalereproductive anatomy
ter Haar Romeny, ICPR 2010
Ovary
Oviduct
Uterus wall Uterus
Endometrium
Uterus neck
ter Haar Romeny, ICPR 2010
The number of follicles decreases during lifetime
ter Haar Romeny, ICPR 2010
1. As female fecundicity decreases with advancing age, an increasing number of couples is faced with unexpected difficulties in conceiving.
• Approx. 15000 couples visit fertility clinics annually• In 70% of these cases age-related fecundicity decline
may play a role• A further increase is expected
2. In our emancipated society a tension between family planning and career exists.
• Being young, till what age can I safely postpone childbearing?• Getting older, at what age am I still likely to be able to conceive
spontaneously?• A further increase is expected
Menopausal age
ter Haar Romeny, ICPR 2010
Resting 0.03 mm initiation of growth> 120 days?
Early growing 0.03 - 0.1 mm
Preantral 0.1 - 0.2 mm basal growth~ 65 days
Antral 0.2 - 2 mm
Selectable 2 - 5 mm rescued by FSH window~ 5 days
Selected 5 - 10 mm
Dominance 10 - 20 mm maturation~ 15 days
Ovulation
A follicle’s life
ter Haar Romeny, ICPR 2010
3D Ultrasound is a safe, cheap and versatile appropriate modality
Kretz Medicor 530D
ter Haar Romeny, ICPR 2010
Two 3D acquisition strategies:
1. Position tracker on regular probe
2. Sweep of 2D array in transducer
Trans-vaginal probe
Regular sampling from irregularly space slices
ter Haar Romeny, ICPR 2010
ter Haar Romeny, ICPR 2010
ter Haar Romeny, ICPR 2010
Manual counting is very cumbersome Automated follicle assessment
• 2-5 mm hypodense structures• structured noise• vessels may look like follicles• ovary boundary sometimes missing
ter Haar Romeny, ICPR 2010
Automated method:
1. Detection of intensity minima by 3D ‘winding numbers’2. Isotropic ray tracing (500 directions) from detected centra3. Edge detection by 1D winding numbers4. Edge focusing to detect most prominent edge5. Fit spherical harmonics to edgepoints6. Calculate follicle shape/size parameters and visualize
ter Haar Romeny, ICPR 2010
Detection of a singularity (i.e. a minimum)
From theory of vector fields several important theorems (Stokes, Gauss) exist that relate something happening in a volume with just its surface.We can detect singularities by measurements around the singularity.
P
1-D: difference of signs of the gradient i zero crossing or extremum
yxi
,
i i
The surrounding of the point P are just 2 pointsleft and right of P 1D sphere.
ter Haar Romeny, ICPR 2010
ijjiW
d
.22
21
1221
dd
dWe consider a unit gradient vector, so 1
2+22=1.
ijjidd In subscript notation:
where ij is the antisymmetric tensor.
01
10ij
ter Haar Romeny, ICPR 2010
For regular points, i.e. when no singularity is present in W, the winding number is zero, as we see from the Stokes’ theorem:
0:21
321 W W
iiiiiii dStokesddd d
d
where the fact that the (d-1)-form is a closed form was used.
So, as most of our datapoints are regular, we detect singularities very robustly as integer values embedded in a space of zero's.
ter Haar Romeny, ICPR 2010
Example of a result:
1 cm
Dataset 2563, radius Stokes’ sphere 1 pixel, blurring scale 3 pixels
ter Haar Romeny, ICPR 2010
• a conservation of winding number within the closed contour.We measure the sum of the winding numbers. E.g. enclosing a saddlepoint and a minimum adds up to zero.
• the winding number is independent of the shape of W.It is a topological entity.
• the winding number only takes integer values.Multiples of the full rotation angle.Eeven when the numerical addition of angles does not sum up to precisely an integer value, we may rightly round off to the nearest integer.
The winding number has nice properties:
ter Haar Romeny, ICPR 2010
• the winding number is a scaled notionThe neighbourhood defines the scale.
• the behaviour over scale generates a tree-like structureTypical annihilations, creations and collisions, from which much can be learned about the ‘deep structure’ of images.
• the winding number is easy to compute, in any dimension.
• the WN is a robust characterisation of the singular points in the image: small deformations have a small effect.
ter Haar Romeny, ICPR 2010
Detection of follicle boundaries:
• generation of 200 - 500 rays in a homogeneous orientation distribution• determine most pronounced edge along ray by winding number focusing• fit spherical harmonics to get an analytical description of the shape• calculate volume and statistics on shape
Distance along rayDistance along ray
Sca
le
US inte
nsi
ty
Sca
le
Distance along ray
ter Haar Romeny, ICPR 2010
3D scatterplot ofdetected endpoints
3D visualisation of fittedspherical harmonics function
ter Haar Romeny, ICPR 2010
ter Haar Romeny, ICPR 2010
Validation with 2 bovine ovaria
• anatomincal coupes• high resolution MR• 3D ultrasound
Follicle#
xcenter
ycenter
zcenter
distance toneighbor(pixels)
volume fromspherical harmonics
(mm3)
volumefrom MRI
volumefrom
anatomyv00 99 51 35 25.4 259.7 250.0 262.1
93 28 44 45.0 28.4 27.0 29.8113 55 74 41.6 56.2 54.9 59.3
v01 33 41 66 25.3 242.347 22 75 44.5 34.064 49 44 38.9 54.7
v44 72 49 84 25.4 239.769 28 70 45.2 28.432 51 82 40.1 59.3
ter Haar Romeny, ICPR 2010
Patient studies:
Performance of the algorithm compared with a human expert. Number of follicles found. Data for 6 patients. The datasets are cut off to contain only the ovary. Scales used: = 3.6, 4.8, 7.2 and 12 pixels.
Patient # manual Computer Patient # Manual computer1 17 15 4 14 92 10 8 5 9 73 7 5 6 9 7
ter Haar Romeny, ICPR 2010
Conclusions:
• 3D ultrasound is a feasible modality for follicle-based fertilitiy state estimation• automated CAD is feasible, more clinical validation needed• winding numbers are robust (scaled) singularity detectors• a robust class of topological properties emerges
ter Haar Romeny, ICPR 2010
Multi-scale watershed segmentation
Watershed are the boundaries of merging water basins, when theimage landscape is immersed by punching the minima.
At larger scale the boundaries get blurred, rounded and dislocated.
ter Haar Romeny, ICPR 2010
Regions of different scales can be linked by calculating thelargest overlap with the region in the scales just above.
ter Haar Romeny, ICPR 2010
The method is often combined with nonlinear diffusion schemes
E. Dam, ITU
ter Haar Romeny, ICPR 2010
Nabla Vision is an interactive 3D watershed segmentation tooldeveloped by the University of Copenhagen.
Sculpture the 3D shape by successively clicking precalculatedfiner scale watershed details.
ter Haar Romeny, ICPR 2010
ter Haar Romeny, ICPR 2010
11
22
1
2 )tan(,tan
We expand the left and right hand side of the last equation in a Taylor series
up to first order in and 1 respectively. For the left hand side we obtain
)(cos
1tan)tan( 2
12
O
).(
)(
212
1
1221
1
2
2112
1
22
1
22
11
22
O
O
3D winding number
And for the righthand side
.21
21
1221
dd
d
ter Haar Romeny, ICPR 2010
In n-D: d
d
iiiiiiiW
ddd 21
321
In 3-D:
))(
)(
)((
dxdz
dzdy
dydx
kzjxkxjz
kyjzkzjy
kxjykyjxiijk
ijkmlkmjli
ijkkji dxdxdd
This expression has to be evaluated for all voxels of our closed surface. We can do this e.g. for the 6 planes of the surrounding cube. On the surface z = constant the previous equation reduces to
dydxkxjykyjxiijk )(
Contraction of indices:
ter Haar Romeny, ICPR 2010
Performing the contraction on the indices i, j and k gives
)(2
)(2
)(2
xyyxyyxxz
zyxxzxxyy
yyzxzyyxx
Calculation of• the gradient vector elements i = { x, y, z} • the derivatives of the gradient field, e.g. x y = y/xis done by neighbour subtraction.The single pixel steps dx and dy are unity.
dydxkxjykyjxiijk )(